2024-04-01T00:03:28.835 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-01T00:03:28.835 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198 --name teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/orig.config.yaml 2024-04-01T00:03:28.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T00:03:29.035 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} email: ceph-qa@ceph.com first_in_suite: false job_id: '7633198' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8238 sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 sleep_before_teardown: 0 subset: 361/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO47IA+EvSR0YgNt5WX/u0uLqP/LctJOxCRWkyqMGKRxSM5C6a6eiCyA2wcSO6kp0CNMI7Qqft9C11I/X8lVYQ= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoLc4FeywrK/SqV54UHq0a0Sj8ZFU63fFNKCIdmYT78xPktjt7ncDU1VQRYkEOPbhv+QsYrsbm8EXPM5SvTa0I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-31_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-01T00:03:29.036 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa; will attempt to use it 2024-04-01T00:03:29.036 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks 2024-04-01T00:03:29.037 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-01T00:03:29.038 INFO:teuthology.task.internal:Checking packages... 2024-04-01T00:03:29.063 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '384f31f3d3aee7f8ba3f60846b8844c049165583' 2024-04-01T00:03:29.063 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-01T00:03:29.063 INFO:teuthology.packaging:ref: None 2024-04-01T00:03:29.063 INFO:teuthology.packaging:tag: None 2024-04-01T00:03:29.063 INFO:teuthology.packaging:branch: squid 2024-04-01T00:03:29.063 INFO:teuthology.packaging:sha1: 384f31f3d3aee7f8ba3f60846b8844c049165583 2024-04-01T00:03:29.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-01T00:03:29.245 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1977.g384f31f3 2024-04-01T00:03:29.247 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-01T00:03:29.310 INFO:teuthology.task.internal:no buildpackages task found 2024-04-01T00:03:29.310 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-01T00:03:29.364 INFO:teuthology.task.internal:Saving configuration 2024-04-01T00:03:29.381 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-01T00:03:29.418 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-01T00:03:29.438 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-31 23:58:26.933271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO47IA+EvSR0YgNt5WX/u0uLqP/LctJOxCRWkyqMGKRxSM5C6a6eiCyA2wcSO6kp0CNMI7Qqft9C11I/X8lVYQ='} 2024-04-01T00:03:29.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-31 23:58:26.934688', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoLc4FeywrK/SqV54UHq0a0Sj8ZFU63fFNKCIdmYT78xPktjt7ncDU1VQRYkEOPbhv+QsYrsbm8EXPM5SvTa0I='} 2024-04-01T00:03:29.462 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-01T00:03:29.473 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-01T00:03:29.473 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-01T00:03:29.473 INFO:teuthology.run_tasks:Running task console_log... 2024-04-01T00:03:29.590 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f076a1f70d0>, signals=[15]) 2024-04-01T00:03:29.591 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-01T00:03:29.613 INFO:teuthology.task.internal:Opening connections... 2024-04-01T00:03:29.613 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:03:29.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:03:29.722 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:03:29.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:03:29.816 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-01T00:03:29.878 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-04-01T00:03:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-04-01T00:03:29.897 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-04-01T00:03:29.958 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-04-01T00:03:29.959 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T00:03:29.959 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-04-01T00:03:29.959 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T00:03:29.959 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T00:03:29.959 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T00:03:29.959 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-04-01T00:03:29.983 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-01T00:03:30.000 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-01T00:03:30.001 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-01T00:03:30.057 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T00:03:30.058 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T00:03:30.059 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T00:03:30.059 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-01T00:03:30.082 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-01T00:03:30.103 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-01T00:03:30.131 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-01T00:03:30.132 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-04-01T00:03:30.134 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-04-01T00:03:30.150 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-01T00:03:30.162 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-01T00:03:30.162 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-04-01T00:03:30.195 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-04-01T00:03:30.267 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-01T00:03:30.332 INFO:teuthology.run_tasks:Running task kernel... 2024-04-01T00:03:30.349 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-01T00:03:30.350 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-01T00:03:30.350 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-01T00:03:30.351 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-01T00:03:30.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:03:30.368 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-01T00:03:30.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:03:30.369 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-04-01T00:03:30.422 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2024-04-01T00:03:30.422 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-361.el9.x86_64 2024-04-01T00:03:30.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-04-01T00:03:30.430 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-361.el9.x86_64 2024-04-01T00:03:30.430 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-361.el9.x86_64 2024-04-01T00:03:30.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-04-01T00:03:32.647 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-01T00:03:32.653 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-01T00:03:34.286 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-04-01T00:03:34.289 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-04-01T00:03:35.329 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-01T00:03:35.449 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-04-01T00:03:40.144 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 19 MB 00:03 2024-04-01T00:03:40.156 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-04-01T00:03:44.586 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.1 MB 00:01 2024-04-01T00:03:45.265 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-04-01T00:03:46.020 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-01T00:03:46.872 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-04-01T00:03:47.640 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-01T00:03:49.794 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-01T00:03:51.922 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-01T00:03:54.128 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-01T00:03:54.129 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-01T00:03:54.218 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:03:54.223 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-04-01T00:03:54.224 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-01T00:03:54.224 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-04-01T00:03:54.224 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:03:54.224 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 76 M 2024-04-01T00:03:54.225 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 123 M 2024-04-01T00:03:54.225 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-04-01T00:03:54.373 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-01T00:03:54.815 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 26 MB/s | 15 MB 00:00 2024-04-01T00:03:55.025 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 51 kB/s | 1.7 kB 00:00 2024-04-01T00:03:55.465 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-core-5.14.0-434.el9.x86_6 25 MB/s | 28 MB 00:01 2024-04-01T00:03:55.866 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 20 MB/s | 34 MB 00:01 2024-04-01T00:03:55.869 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-04-01T00:03:55.869 INFO:teuthology.orchestra.run.smithi100.stdout:Total 46 MB/s | 76 MB 00:01 2024-04-01T00:03:55.869 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-04-01T00:03:56.245 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-04-01T00:03:56.246 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-04-01T00:03:57.086 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-04-01T00:03:57.087 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-04-01T00:03:57.348 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-01T00:03:57.349 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-01T00:03:57.448 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:03:57.455 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-04-01T00:03:57.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:03:57.456 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 76 M 2024-04-01T00:03:57.456 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 123 M 2024-04-01T00:03:57.456 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-04-01T00:03:57.618 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-01T00:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-04-01T00:03:58.818 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-core-5.14.0-434.el9.x86_6 23 MB/s | 28 MB 00:01 2024-04-01T00:03:59.119 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 20 MB/s | 34 MB 00:01 2024-04-01T00:03:59.122 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-04-01T00:03:59.122 INFO:teuthology.orchestra.run.smithi064.stdout:Total 46 MB/s | 76 MB 00:01 2024-04-01T00:03:59.122 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-04-01T00:03:59.436 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-04-01T00:03:59.553 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-04-01T00:03:59.553 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-04-01T00:04:00.114 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-01T00:04:00.202 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:00.433 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-04-01T00:04:00.433 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-04-01T00:04:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-04-01T00:04:02.934 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:03.071 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-01T00:04:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:06.429 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:07.804 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:07.853 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:11.630 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:11.668 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:12.601 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:16.771 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:41.870 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:41.871 INFO:teuthology.orchestra.run.smithi100.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-01T00:04:41.871 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:04:41.913 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:42.250 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:42.250 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-01T00:04:42.250 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:42.250 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:42.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:42.656 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:04:42.656 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-04-01T00:04:42.656 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:42.656 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-01T00:04:42.657 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-01T00:04:42.657 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-01T00:04:42.657 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:04:42.657 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-04-01T00:04:42.996 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-04-01T00:04:43.090 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:43.091 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-01T00:04:43.091 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-01T00:04:43.091 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-01T00:04:43.091 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-01T00:04:43.091 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-01T00:04:43.092 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-04-01T00:04:43.092 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-04-01T00:04:43.172 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:43.172 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-01T00:04:43.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-04-01T00:04:43.208 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-04-01T00:04:43.208 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-04-01T00:04:43.208 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-01T00:04:43.209 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:04:43.210 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-04-01T00:04:43.211 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-01T00:04:43.211 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-434.el9.x86_64 2024-04-01T00:04:43.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-01T00:04:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-04-01T00:04:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-01T00:04:45.245 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-04-01T00:04:45.247 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-01T00:04:45.262 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.fvGmxMmlqh 2024-04-01T00:04:45.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fvGmxMmlqh 2024-04-01T00:04:45.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.fvGmxMmlqh 2024-04-01T00:04:45.460 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.fvGmxMmlqh is 6KB 2024-04-01T00:04:45.508 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.fvGmxMmlqh 2024-04-01T00:04:45.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-04-01T00:04:45.590 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-01T00:04:45.590 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-01T00:04:45.590 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-01T00:04:45.590 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-01T00:04:45.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-01T00:04:45.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-04-01T00:04:47.828 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:47.828 INFO:teuthology.orchestra.run.smithi064.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-01T00:04:47.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:04:47.873 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:48.248 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:48.249 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-01T00:04:48.249 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-01T00:04:48.249 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-01T00:04:48.699 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-01T00:04:48.699 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:04:48.700 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-04-01T00:04:49.041 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-04-01T00:04:49.181 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:49.182 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-01T00:04:49.182 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-01T00:04:49.183 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-01T00:04:49.183 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-01T00:04:49.183 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-01T00:04:49.183 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-04-01T00:04:49.183 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-04-01T00:04:49.267 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-01T00:04:49.267 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-01T00:04:49.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-04-01T00:04:49.313 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-04-01T00:04:49.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:04:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-04-01T00:04:49.316 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-01T00:04:49.316 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-434.el9.x86_64 2024-04-01T00:04:49.316 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-01T00:04:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-04-01T00:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-01T00:04:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-04-01T00:04:51.377 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-04-01T00:04:51.432 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.J04Yl1Bwkj 2024-04-01T00:04:51.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.J04Yl1Bwkj 2024-04-01T00:04:51.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.J04Yl1Bwkj 2024-04-01T00:04:51.652 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.J04Yl1Bwkj is 6KB 2024-04-01T00:04:51.700 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.J04Yl1Bwkj 2024-04-01T00:04:51.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-04-01T00:04:51.793 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-01T00:04:51.793 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-01T00:04:51.793 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-01T00:04:51.793 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-01T00:04:51.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-01T00:04:51.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-04-01T00:05:15.787 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-01T00:05:15.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:05:15.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:05:21.950 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-01T00:05:21.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:05:21.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:05:34.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-01T00:05:40.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-04-01T00:05:43.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:05:43.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:05:46.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-01T00:05:49.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:05:49.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:05:52.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-04-01T00:05:58.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:05:58.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:06:04.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:06:04.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:06:30.088 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-01T00:06:30.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:06:30.505 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-434.el9.x86_64"... 2024-04-01T00:06:30.506 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-01T00:06:30.519 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-434.el9.x86_64 2024-04-01T00:06:30.519 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-01T00:06:30.519 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-01T00:06:30.520 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-01T00:06:31.521 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-01T00:06:31.522 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-01T00:06:31.611 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-04-01T00:06:31.649 DEBUG:teuthology.parallel:result is None 2024-04-01T00:06:36.478 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-04-01T00:06:37.179 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:06:37.180 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-434.el9.x86_64"... 2024-04-01T00:06:37.180 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-04-01T00:06:37.194 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-434.el9.x86_64 2024-04-01T00:06:37.194 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-01T00:06:37.194 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-01T00:06:37.194 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-01T00:06:38.195 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-01T00:06:38.195 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-01T00:06:38.302 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-04-01T00:06:38.361 DEBUG:teuthology.parallel:result is None 2024-04-01T00:06:38.361 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-01T00:06:38.368 INFO:teuthology.task.internal:Creating test directory... 2024-04-01T00:06:38.368 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-01T00:06:38.370 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-01T00:06:38.393 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-01T00:06:38.400 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-01T00:06:38.437 INFO:teuthology.task.internal:Creating archive directory... 2024-04-01T00:06:38.438 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-01T00:06:38.440 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-01T00:06:38.477 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-01T00:06:38.484 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-01T00:06:38.485 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-01T00:06:38.518 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-01T00:06:38.553 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-01T00:06:38.566 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-01T00:06:38.575 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-01T00:06:38.599 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-01T00:06:38.602 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-01T00:06:38.609 INFO:teuthology.task.internal:Configuring sudo... 2024-04-01T00:06:38.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-01T00:06:38.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-01T00:06:38.654 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-01T00:06:38.667 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-01T00:06:38.667 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-01T00:06:38.700 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-01T00:06:38.722 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-01T00:06:38.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-01T00:06:38.890 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-01T00:06:38.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-01T00:06:38.988 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:06:38.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-01T00:06:39.059 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-01T00:06:39.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-01T00:06:39.156 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-01T00:06:39.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-01T00:06:39.287 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:06:39.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-01T00:06:39.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-04-01T00:06:39.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-04-01T00:06:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-01T00:06:39.473 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-01T00:06:39.985 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-01T00:06:40.010 INFO:teuthology.task.internal:Starting timer... 2024-04-01T00:06:40.011 INFO:teuthology.run_tasks:Running task pcp... 2024-04-01T00:06:40.047 INFO:teuthology.run_tasks:Running task selinux... 2024-04-01T00:06:40.081 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-01T00:06:40.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-04-01T00:06:40.129 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-04-01T00:06:40.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-04-01T00:06:40.200 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-04-01T00:06:40.223 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-01T00:06:40.223 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-04-01T00:06:40.242 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-04-01T00:06:40.242 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-04-01T00:06:40.261 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-04-01T00:06:40.262 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2024-04-01T00:06:40.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-01T00:06:40.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:06:40.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-01T00:06:40.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:06:40.338 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-01T00:06:40.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-04-01T00:06:40.386 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-04-01T00:06:40.422 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-01T00:06:40.466 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-01T00:06:40.481 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-01T00:06:40.483 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-01T00:10:53.586 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-04-01T00:10:53.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:10:53.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:10:53.678 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-04-01T00:10:53.759 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-01T00:10:53.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:10:53.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T00:10:53.851 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-01T00:10:53.932 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-01T00:10:53.932 INFO:teuthology.run_tasks:Running task clock... 2024-04-01T00:10:53.942 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-01T00:10:53.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-01T00:10:53.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-01T00:10:53.945 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-01T00:10:53.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-01T00:10:53.975 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-01T00:10:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-01T00:10:54.002 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-01T00:10:54.018 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-01T00:10:54.022 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-04-01T00:10:54.036 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-04-01T00:10:54.047 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-04-01T00:10:54.053 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-01T00:10:54.061 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-04-01T00:10:54.069 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-01T00:10:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-01T00:10:54.094 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-01T00:10:54.123 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.126 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.150 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T00:10:54.151 INFO:teuthology.run_tasks:Running task install... 2024-04-01T00:10:54.166 DEBUG:teuthology.task.install:project ceph 2024-04-01T00:10:54.167 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'}} 2024-04-01T00:10:54.167 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'} 2024-04-01T00:10:54.167 INFO:teuthology.task.install:Using flavor: default 2024-04-01T00:10:54.173 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-01T00:10:54.173 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-01T00:10:54.173 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583', 'tag': None, 'wait_for_package': False} 2024-04-01T00:10:54.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384f31f3d3aee7f8ba3f60846b8844c049165583 2024-04-01T00:10:54.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583', 'tag': None, 'wait_for_package': False} 2024-04-01T00:10:54.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384f31f3d3aee7f8ba3f60846b8844c049165583 2024-04-01T00:10:54.322 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/ 2024-04-01T00:10:54.322 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1977.g384f31f3 2024-04-01T00:10:54.391 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/ 2024-04-01T00:10:54.391 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1977.g384f31f3 2024-04-01T00:10:54.496 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-01T00:10:54.497 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:10:54.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-01T00:10:54.501 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/384f31f3d3aee7f8ba3f60846b8844c049165583/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-01T00:10:54.501 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:10:54.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-01T00:10:54.525 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-01T00:10:54.526 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384f31f3d3aee7f8ba3f60846b8844c049165583/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-01T00:10:54.530 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-01T00:10:54.530 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384f31f3d3aee7f8ba3f60846b8844c049165583/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-01T00:10:54.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-01T00:10:54.598 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-01T00:10:54.673 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-01T00:10:54.677 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-01T00:10:54.739 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-04-01T00:10:54.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-04-01T00:10:54.744 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-04-01T00:10:54.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-04-01T00:10:54.983 INFO:teuthology.orchestra.run.smithi064.stdout:74 files removed 2024-04-01T00:10:54.990 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-04-01T00:10:55.010 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-01T00:10:55.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-01T00:10:55.715 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 208 kB/s | 83 kB 00:00 2024-04-01T00:10:55.781 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-04-01T00:10:56.141 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-04-01T00:10:56.156 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-01T00:10:56.533 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-01T00:10:56.540 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-01T00:10:56.959 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-01T00:10:57.111 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-01T00:10:58.410 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-04-01T00:10:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-04-01T00:10:59.536 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-01T00:10:59.676 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-01T00:11:01.335 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-01T00:11:06.702 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 19 MB 00:05 2024-04-01T00:11:12.768 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 805 kB/s | 6.1 MB 00:07 2024-04-01T00:11:14.420 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-01T00:11:15.464 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-01T00:11:17.885 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 817 kB/s | 6.1 MB 00:07 2024-04-01T00:11:20.169 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-01T00:11:20.639 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-01T00:11:22.606 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-01T00:11:23.286 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-01T00:11:23.287 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-01T00:11:23.300 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-01T00:11:23.301 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-01T00:11:23.301 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-01T00:11:23.302 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-01T00:11:23.304 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-01T00:11:23.307 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-01T00:11:23.308 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-01T00:11:23.394 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-04-01T00:11:23.415 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-04-01T00:11:23.415 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 6.6 k 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 5.3 M 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 889 k 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 145 k 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 1.5 M 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 139 k 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 3.5 M 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 7.4 M 2024-04-01T00:11:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 49 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 9.8 M 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 47 M 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 264 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 748 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 221 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 27 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 18 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 32 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 746 k 2024-04-01T00:11:23.417 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 127 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 161 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 320 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 298 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 99 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 85 k 2024-04-01T00:11:23.418 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.1 M 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 171 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-01T00:11:23.419 INFO:teuthology.orchestra.run.smithi064.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.4 M 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.1 M 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-01T00:11:23.420 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 21 M 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 31 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 2.4 M 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 248 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 4.7 M 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 17 M 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 15 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 25 k 2024-04-01T00:11:23.421 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 163 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-01T00:11:23.422 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 501 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 5.1 M 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-01T00:11:23.423 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 45 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 131 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-01T00:11:23.424 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-01T00:11:23.425 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-01T00:11:23.426 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-01T00:11:23.427 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-01T00:11:23.428 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout:Install 150 Packages 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 24 Packages 2024-04-01T00:11:23.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:11:23.430 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-04-01T00:11:23.430 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-04-01T00:11:23.899 INFO:teuthology.orchestra.run.smithi064.stdout:(1/174): ceph-19.0.0-1977.g384f31f3.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-04-01T00:11:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:(2/174): ceph-fuse-19.0.0-1977.g384f31f3.el9.x8 3.7 MB/s | 889 kB 00:00 2024-04-01T00:11:24.191 INFO:teuthology.orchestra.run.smithi064.stdout:(3/174): ceph-immutable-object-cache-19.0.0-197 2.4 MB/s | 145 kB 00:00 2024-04-01T00:11:24.383 INFO:teuthology.orchestra.run.smithi064.stdout:(4/174): ceph-base-19.0.0-1977.g384f31f3.el9.x8 8.3 MB/s | 5.3 MB 00:00 2024-04-01T00:11:24.492 INFO:teuthology.orchestra.run.smithi064.stdout:(5/174): ceph-mds-19.0.0-1977.g384f31f3.el9.x86 7.8 MB/s | 2.4 MB 00:00 2024-04-01T00:11:24.542 INFO:teuthology.orchestra.run.smithi064.stdout:(6/174): ceph-mgr-19.0.0-1977.g384f31f3.el9.x86 9.4 MB/s | 1.5 MB 00:00 2024-04-01T00:11:24.958 INFO:teuthology.orchestra.run.smithi064.stdout:(7/174): ceph-mon-19.0.0-1977.g384f31f3.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-01T00:11:25.684 INFO:teuthology.orchestra.run.smithi064.stdout:(8/174): ceph-osd-19.0.0-1977.g384f31f3.el9.x86 15 MB/s | 17 MB 00:01 2024-04-01T00:11:25.751 INFO:teuthology.orchestra.run.smithi064.stdout:(9/174): ceph-selinux-19.0.0-1977.g384f31f3.el9 378 kB/s | 25 kB 00:00 2024-04-01T00:11:25.876 INFO:teuthology.orchestra.run.smithi064.stdout:(10/174): ceph-radosgw-19.0.0-1977.g384f31f3.el 11 MB/s | 9.8 MB 00:00 2024-04-01T00:11:25.985 INFO:teuthology.orchestra.run.smithi064.stdout:(11/174): cephfs-mirror-19.0.0-1977.g384f31f3.e 2.0 MB/s | 221 kB 00:00 2024-04-01T00:11:26.068 INFO:teuthology.orchestra.run.smithi064.stdout:(12/174): cephfs-shell-19.0.0-1977.g384f31f3.el 324 kB/s | 27 kB 00:00 2024-04-01T00:11:26.160 INFO:teuthology.orchestra.run.smithi064.stdout:(13/174): libcephfs-devel-19.0.0-1977.g384f31f3 347 kB/s | 32 kB 00:00 2024-04-01T00:11:26.294 INFO:teuthology.orchestra.run.smithi064.stdout:(14/174): libcephfs2-19.0.0-1977.g384f31f3.el9. 5.5 MB/s | 746 kB 00:00 2024-04-01T00:11:26.486 INFO:teuthology.orchestra.run.smithi064.stdout:(15/174): libcephsqlite-19.0.0-1977.g384f31f3.e 851 kB/s | 163 kB 00:00 2024-04-01T00:11:26.544 INFO:teuthology.orchestra.run.smithi064.stdout:(16/174): librados-devel-19.0.0-1977.g384f31f3. 2.1 MB/s | 127 kB 00:00 2024-04-01T00:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:(17/174): libradosstriper1-19.0.0-1977.g384f31f 5.9 MB/s | 501 kB 00:00 2024-04-01T00:11:27.161 INFO:teuthology.orchestra.run.smithi064.stdout:(18/174): librgw2-19.0.0-1977.g384f31f3.el9.x86 9.6 MB/s | 5.1 MB 00:00 2024-04-01T00:11:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:(19/174): python3-ceph-argparse-19.0.0-1977.g38 678 kB/s | 45 kB 00:00 2024-04-01T00:11:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:(20/174): python3-ceph-common-19.0.0-1977.g384f 1.7 MB/s | 131 kB 00:00 2024-04-01T00:11:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:(21/174): python3-cephfs-19.0.0-1977.g384f31f3. 2.1 MB/s | 161 kB 00:00 2024-04-01T00:11:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:(22/174): python3-rados-19.0.0-1977.g384f31f3.e 3.8 MB/s | 320 kB 00:00 2024-04-01T00:11:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:(23/174): python3-rbd-19.0.0-1977.g384f31f3.el9 3.5 MB/s | 298 kB 00:00 2024-04-01T00:11:27.613 INFO:teuthology.orchestra.run.smithi064.stdout:(24/174): python3-rgw-19.0.0-1977.g384f31f3.el9 1.5 MB/s | 99 kB 00:00 2024-04-01T00:11:27.653 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-01T00:11:27.680 INFO:teuthology.orchestra.run.smithi064.stdout:(25/174): rbd-fuse-19.0.0-1977.g384f31f3.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-01T00:11:28.072 INFO:teuthology.orchestra.run.smithi064.stdout:(26/174): rbd-mirror-19.0.0-1977.g384f31f3.el9. 7.9 MB/s | 3.1 MB 00:00 2024-04-01T00:11:28.172 INFO:teuthology.orchestra.run.smithi064.stdout:(27/174): rbd-nbd-19.0.0-1977.g384f31f3.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-01T00:11:28.248 INFO:teuthology.orchestra.run.smithi064.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1977.g 420 kB/s | 31 kB 00:00 2024-04-01T00:11:28.331 INFO:teuthology.orchestra.run.smithi064.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1977.g384f31f 1.6 MB/s | 139 kB 00:00 2024-04-01T00:11:28.598 INFO:teuthology.orchestra.run.smithi064.stdout:(30/174): ceph-common-19.0.0-1977.g384f31f3.el9 4.2 MB/s | 21 MB 00:04 2024-04-01T00:11:28.690 INFO:teuthology.orchestra.run.smithi064.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1977.g384f3 9.8 MB/s | 3.5 MB 00:00 2024-04-01T00:11:28.782 INFO:teuthology.orchestra.run.smithi064.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1977.g38 2.7 MB/s | 248 kB 00:00 2024-04-01T00:11:28.874 INFO:teuthology.orchestra.run.smithi064.stdout:(33/174): ceph-mgr-rook-19.0.0-1977.g384f31f3.e 538 kB/s | 49 kB 00:00 2024-04-01T00:11:29.308 INFO:teuthology.orchestra.run.smithi064.stdout:(34/174): ceph-test-19.0.0-1977.g384f31f3.el9.x 13 MB/s | 47 MB 00:03 2024-04-01T00:11:29.333 INFO:teuthology.orchestra.run.smithi064.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1977.g3 33 kB/s | 15 kB 00:00 2024-04-01T00:11:29.358 INFO:teuthology.orchestra.run.smithi064.stdout:(36/174): ceph-volume-19.0.0-1977.g384f31f3.el9 5.2 MB/s | 264 kB 00:00 2024-04-01T00:11:29.417 INFO:teuthology.orchestra.run.smithi064.stdout:(37/174): cephadm-19.0.0-1977.g384f31f3.el9.noa 8.8 MB/s | 748 kB 00:00 2024-04-01T00:11:29.442 INFO:teuthology.orchestra.run.smithi064.stdout:(38/174): cephfs-top-19.0.0-1977.g384f31f3.el9. 218 kB/s | 18 kB 00:00 2024-04-01T00:11:29.525 INFO:teuthology.orchestra.run.smithi064.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-04-01T00:11:29.684 INFO:teuthology.orchestra.run.smithi064.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2024-04-01T00:11:29.726 INFO:teuthology.orchestra.run.smithi064.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-04-01T00:11:29.768 INFO:teuthology.orchestra.run.smithi064.stdout:(42/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-04-01T00:11:29.802 INFO:teuthology.orchestra.run.smithi064.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2024-04-01T00:11:29.818 INFO:teuthology.orchestra.run.smithi064.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 344 kB/s | 32 kB 00:00 2024-04-01T00:11:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 17 kB 00:00 2024-04-01T00:11:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-01T00:11:29.985 INFO:teuthology.orchestra.run.smithi064.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 60 kB/s | 10 kB 00:00 2024-04-01T00:11:30.011 INFO:teuthology.orchestra.run.smithi064.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-04-01T00:11:30.036 INFO:teuthology.orchestra.run.smithi064.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-04-01T00:11:30.070 INFO:teuthology.orchestra.run.smithi064.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-04-01T00:11:30.104 INFO:teuthology.orchestra.run.smithi064.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-04-01T00:11:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 509 kB/s | 46 kB 00:00 2024-04-01T00:11:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-04-01T00:11:30.179 INFO:teuthology.orchestra.run.smithi064.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-01T00:11:30.205 INFO:teuthology.orchestra.run.smithi064.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-01T00:11:30.206 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-01T00:11:30.207 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-01T00:11:30.221 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-01T00:11:30.222 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-01T00:11:30.223 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-01T00:11:30.224 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-01T00:11:30.226 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-01T00:11:30.229 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-01T00:11:30.230 INFO:teuthology.orchestra.run.smithi064.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 455 kB/s | 34 kB 00:00 2024-04-01T00:11:30.230 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-01T00:11:30.309 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 6.6 k 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 5.3 M 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 889 k 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 145 k 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 1.5 M 2024-04-01T00:11:30.327 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 139 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 3.5 M 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 7.4 M 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 49 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 9.8 M 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 47 M 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 264 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 748 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 221 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 27 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 18 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-01T00:11:30.328 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 32 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 746 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 127 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 161 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 320 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 298 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 99 k 2024-04-01T00:11:30.329 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 85 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.1 M 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 171 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-01T00:11:30.330 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.4 M 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 3.1 M 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-01T00:11:30.331 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 21 M 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 31 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 2.4 M 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 248 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 4.7 M 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 17 M 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1977.g384f31f3.el9 ceph-noarch 15 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 25 k 2024-04-01T00:11:30.332 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 163 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-01T00:11:30.333 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 501 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 5.1 M 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-01T00:11:30.334 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 45 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-1977.g384f31f3.el9 ceph 131 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-01T00:11:30.335 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-01T00:11:30.336 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-01T00:11:30.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-01T00:11:30.338 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-01T00:11:30.339 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 24 Packages 2024-04-01T00:11:30.340 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:11:30.341 INFO:teuthology.orchestra.run.smithi064.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-04-01T00:11:30.341 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 226 M 2024-04-01T00:11:30.341 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-04-01T00:11:30.430 INFO:teuthology.orchestra.run.smithi064.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 817 kB 00:00 2024-04-01T00:11:30.497 INFO:teuthology.orchestra.run.smithi064.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-01T00:11:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 697 kB/s | 132 kB 00:00 2024-04-01T00:11:30.631 INFO:teuthology.orchestra.run.smithi064.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2024-04-01T00:11:30.656 INFO:teuthology.orchestra.run.smithi064.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 597 kB/s | 78 kB 00:00 2024-04-01T00:11:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-04-01T00:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-04-01T00:11:30.930 INFO:teuthology.orchestra.run.smithi100.stdout:(1/174): ceph-19.0.0-1977.g384f31f3.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-01T00:11:30.966 INFO:teuthology.orchestra.run.smithi064.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 80 kB/s | 24 kB 00:00 2024-04-01T00:11:30.991 INFO:teuthology.orchestra.run.smithi064.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 61 kB/s | 18 kB 00:00 2024-04-01T00:11:31.016 INFO:teuthology.orchestra.run.smithi064.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-04-01T00:11:31.041 INFO:teuthology.orchestra.run.smithi064.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-04-01T00:11:31.117 INFO:teuthology.orchestra.run.smithi064.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-04-01T00:11:31.138 INFO:teuthology.orchestra.run.smithi100.stdout:(2/174): ceph-fuse-19.0.0-1977.g384f31f3.el9.x8 4.2 MB/s | 889 kB 00:00 2024-04-01T00:11:31.142 INFO:teuthology.orchestra.run.smithi064.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-04-01T00:11:31.167 INFO:teuthology.orchestra.run.smithi064.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-04-01T00:11:31.188 INFO:teuthology.orchestra.run.smithi100.stdout:(3/174): ceph-immutable-object-cache-19.0.0-197 2.8 MB/s | 145 kB 00:00 2024-04-01T00:11:31.193 INFO:teuthology.orchestra.run.smithi064.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-04-01T00:11:31.217 INFO:teuthology.orchestra.run.smithi064.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-04-01T00:11:31.243 INFO:teuthology.orchestra.run.smithi064.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-01T00:11:31.268 INFO:teuthology.orchestra.run.smithi064.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-01T00:11:31.293 INFO:teuthology.orchestra.run.smithi064.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 955 kB/s | 47 kB 00:00 2024-04-01T00:11:31.318 INFO:teuthology.orchestra.run.smithi064.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-01T00:11:31.344 INFO:teuthology.orchestra.run.smithi064.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-04-01T00:11:31.464 INFO:teuthology.orchestra.run.smithi100.stdout:(4/174): ceph-mds-19.0.0-1977.g384f31f3.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-01T00:11:31.606 INFO:teuthology.orchestra.run.smithi100.stdout:(5/174): ceph-mgr-19.0.0-1977.g384f31f3.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-01T00:11:31.735 INFO:teuthology.orchestra.run.smithi064.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-01T00:11:31.752 INFO:teuthology.orchestra.run.smithi064.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 95 kB/s | 39 kB 00:00 2024-04-01T00:11:31.769 INFO:teuthology.orchestra.run.smithi064.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 97 kB/s | 41 kB 00:00 2024-04-01T00:11:31.803 INFO:teuthology.orchestra.run.smithi064.stdout:(82/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-01T00:11:31.828 INFO:teuthology.orchestra.run.smithi064.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 600 kB/s | 35 kB 00:00 2024-04-01T00:11:31.853 INFO:teuthology.orchestra.run.smithi064.stdout:(84/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2024-04-01T00:11:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-01T00:11:31.912 INFO:teuthology.orchestra.run.smithi064.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 676 kB/s | 56 kB 00:00 2024-04-01T00:11:31.937 INFO:teuthology.orchestra.run.smithi064.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 746 kB/s | 63 kB 00:00 2024-04-01T00:11:31.963 INFO:teuthology.orchestra.run.smithi064.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-04-01T00:11:31.988 INFO:teuthology.orchestra.run.smithi064.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-01T00:11:32.013 INFO:teuthology.orchestra.run.smithi064.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-01T00:11:32.014 INFO:teuthology.orchestra.run.smithi100.stdout:(6/174): ceph-mon-19.0.0-1977.g384f31f3.el9.x86 11 MB/s | 4.7 MB 00:00 2024-04-01T00:11:32.055 INFO:teuthology.orchestra.run.smithi064.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-01T00:11:32.080 INFO:teuthology.orchestra.run.smithi064.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-01T00:11:32.089 INFO:teuthology.orchestra.run.smithi100.stdout:(7/174): ceph-base-19.0.0-1977.g384f31f3.el9.x8 4.0 MB/s | 5.3 MB 00:01 2024-04-01T00:11:32.106 INFO:teuthology.orchestra.run.smithi064.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-04-01T00:11:32.131 INFO:teuthology.orchestra.run.smithi064.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-04-01T00:11:32.248 INFO:teuthology.orchestra.run.smithi064.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-01T00:11:32.273 INFO:teuthology.orchestra.run.smithi064.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-01T00:11:32.299 INFO:teuthology.orchestra.run.smithi064.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-01T00:11:32.332 INFO:teuthology.orchestra.run.smithi064.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-01T00:11:32.407 INFO:teuthology.orchestra.run.smithi064.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-04-01T00:11:32.424 INFO:teuthology.orchestra.run.smithi064.stdout:(100/174): elfutils-libelf-devel-0.190-2.el9.x8 143 kB/s | 21 kB 00:00 2024-04-01T00:11:32.583 INFO:teuthology.orchestra.run.smithi064.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-04-01T00:11:32.608 INFO:teuthology.orchestra.run.smithi064.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-04-01T00:11:32.675 INFO:teuthology.orchestra.run.smithi064.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-04-01T00:11:32.892 INFO:teuthology.orchestra.run.smithi064.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-01T00:11:33.342 INFO:teuthology.orchestra.run.smithi064.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-01T00:11:33.376 INFO:teuthology.orchestra.run.smithi064.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-01T00:11:33.435 INFO:teuthology.orchestra.run.smithi064.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.7 MB/s | 4.1 MB 00:01 2024-04-01T00:11:33.485 INFO:teuthology.orchestra.run.smithi064.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-04-01T00:11:33.548 INFO:teuthology.orchestra.run.smithi100.stdout:(8/174): ceph-osd-19.0.0-1977.g384f31f3.el9.x86 11 MB/s | 17 MB 00:01 2024-04-01T00:11:33.560 INFO:teuthology.orchestra.run.smithi064.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-04-01T00:11:33.635 INFO:teuthology.orchestra.run.smithi064.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-01T00:11:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:(111/174): grpc-data-1.46.7-10.el9.noarch.rpm 68 kB/s | 19 kB 00:00 2024-04-01T00:11:33.711 INFO:teuthology.orchestra.run.smithi064.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-01T00:11:33.748 INFO:teuthology.orchestra.run.smithi100.stdout:(9/174): ceph-common-19.0.0-1977.g384f31f3.el9. 6.9 MB/s | 21 MB 00:02 2024-04-01T00:11:33.773 INFO:teuthology.orchestra.run.smithi100.stdout:(10/174): ceph-selinux-19.0.0-1977.g384f31f3.el 112 kB/s | 25 kB 00:00 2024-04-01T00:11:33.836 INFO:teuthology.orchestra.run.smithi064.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-01T00:11:33.861 INFO:teuthology.orchestra.run.smithi064.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 753 kB/s | 151 kB 00:00 2024-04-01T00:11:33.899 INFO:teuthology.orchestra.run.smithi100.stdout:(11/174): ceph-radosgw-19.0.0-1977.g384f31f3.el 5.4 MB/s | 9.8 MB 00:01 2024-04-01T00:11:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-01T00:11:33.924 INFO:teuthology.orchestra.run.smithi100.stdout:(12/174): cephfs-mirror-19.0.0-1977.g384f31f3.e 1.4 MB/s | 221 kB 00:00 2024-04-01T00:11:33.945 INFO:teuthology.orchestra.run.smithi064.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-01T00:11:33.966 INFO:teuthology.orchestra.run.smithi100.stdout:(13/174): cephfs-shell-19.0.0-1977.g384f31f3.el 406 kB/s | 27 kB 00:00 2024-04-01T00:11:33.987 INFO:teuthology.orchestra.run.smithi064.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-04-01T00:11:33.991 INFO:teuthology.orchestra.run.smithi100.stdout:(14/174): libcephfs-devel-19.0.0-1977.g384f31f3 473 kB/s | 32 kB 00:00 2024-04-01T00:11:34.012 INFO:teuthology.orchestra.run.smithi064.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-04-01T00:11:34.050 INFO:teuthology.orchestra.run.smithi100.stdout:(15/174): libcephsqlite-19.0.0-1977.g384f31f3.e 2.7 MB/s | 163 kB 00:00 2024-04-01T00:11:34.063 INFO:teuthology.orchestra.run.smithi064.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-01T00:11:34.075 INFO:teuthology.orchestra.run.smithi100.stdout:(16/174): libcephfs2-19.0.0-1977.g384f31f3.el9. 6.7 MB/s | 746 kB 00:00 2024-04-01T00:11:34.108 INFO:teuthology.orchestra.run.smithi100.stdout:(17/174): librados-devel-19.0.0-1977.g384f31f3. 2.1 MB/s | 127 kB 00:00 2024-04-01T00:11:34.138 INFO:teuthology.orchestra.run.smithi064.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-01T00:11:34.158 INFO:teuthology.orchestra.run.smithi100.stdout:(18/174): libradosstriper1-19.0.0-1977.g384f31f 5.9 MB/s | 501 kB 00:00 2024-04-01T00:11:34.163 INFO:teuthology.orchestra.run.smithi064.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-01T00:11:34.205 INFO:teuthology.orchestra.run.smithi064.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-01T00:11:34.239 INFO:teuthology.orchestra.run.smithi064.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-01T00:11:34.262 INFO:teuthology.orchestra.run.smithi100.stdout:(19/174): python3-ceph-argparse-19.0.0-1977.g38 450 kB/s | 45 kB 00:00 2024-04-01T00:11:34.264 INFO:teuthology.orchestra.run.smithi064.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-01T00:11:34.314 INFO:teuthology.orchestra.run.smithi064.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-01T00:11:34.339 INFO:teuthology.orchestra.run.smithi064.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-01T00:11:34.359 INFO:teuthology.orchestra.run.smithi100.stdout:(20/174): python3-ceph-common-19.0.0-1977.g384f 1.3 MB/s | 131 kB 00:00 2024-04-01T00:11:34.381 INFO:teuthology.orchestra.run.smithi064.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-01T00:11:34.406 INFO:teuthology.orchestra.run.smithi064.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-01T00:11:34.451 INFO:teuthology.orchestra.run.smithi100.stdout:(21/174): python3-cephfs-19.0.0-1977.g384f31f3. 1.7 MB/s | 161 kB 00:00 2024-04-01T00:11:34.515 INFO:teuthology.orchestra.run.smithi064.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-01T00:11:34.518 INFO:teuthology.orchestra.run.smithi100.stdout:(22/174): librgw2-19.0.0-1977.g384f31f3.el9.x86 13 MB/s | 5.1 MB 00:00 2024-04-01T00:11:34.540 INFO:teuthology.orchestra.run.smithi064.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 122 kB/s | 19 kB 00:00 2024-04-01T00:11:34.543 INFO:teuthology.orchestra.run.smithi100.stdout:(23/174): python3-rados-19.0.0-1977.g384f31f3.e 3.4 MB/s | 320 kB 00:00 2024-04-01T00:11:34.557 INFO:teuthology.orchestra.run.smithi064.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-04-01T00:11:34.586 INFO:teuthology.orchestra.run.smithi100.stdout:(24/174): python3-rbd-19.0.0-1977.g384f31f3.el9 4.4 MB/s | 298 kB 00:00 2024-04-01T00:11:34.608 INFO:teuthology.orchestra.run.smithi064.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-01T00:11:34.633 INFO:teuthology.orchestra.run.smithi064.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-01T00:11:34.644 INFO:teuthology.orchestra.run.smithi100.stdout:(25/174): python3-rgw-19.0.0-1977.g384f31f3.el9 994 kB/s | 99 kB 00:00 2024-04-01T00:11:34.658 INFO:teuthology.orchestra.run.smithi064.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-01T00:11:34.669 INFO:teuthology.orchestra.run.smithi100.stdout:(26/174): rbd-fuse-19.0.0-1977.g384f31f3.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-01T00:11:34.683 INFO:teuthology.orchestra.run.smithi064.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-01T00:11:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-01T00:11:34.725 INFO:teuthology.orchestra.run.smithi064.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-01T00:11:34.728 INFO:teuthology.orchestra.run.smithi100.stdout:(27/174): rbd-nbd-19.0.0-1977.g384f31f3.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-01T00:11:34.751 INFO:teuthology.orchestra.run.smithi064.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-01T00:11:34.768 INFO:teuthology.orchestra.run.smithi064.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-04-01T00:11:34.785 INFO:teuthology.orchestra.run.smithi064.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 521 kB/s | 31 kB 00:00 2024-04-01T00:11:34.786 INFO:teuthology.orchestra.run.smithi100.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1977.g 540 kB/s | 31 kB 00:00 2024-04-01T00:11:34.818 INFO:teuthology.orchestra.run.smithi064.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-01T00:11:34.835 INFO:teuthology.orchestra.run.smithi064.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-04-01T00:11:34.852 INFO:teuthology.orchestra.run.smithi064.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-04-01T00:11:34.862 INFO:teuthology.orchestra.run.smithi100.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1977.g384f31f 1.8 MB/s | 139 kB 00:00 2024-04-01T00:11:34.877 INFO:teuthology.orchestra.run.smithi064.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-04-01T00:11:34.911 INFO:teuthology.orchestra.run.smithi064.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-01T00:11:34.936 INFO:teuthology.orchestra.run.smithi064.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-01T00:11:34.970 INFO:teuthology.orchestra.run.smithi064.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-01T00:11:34.995 INFO:teuthology.orchestra.run.smithi064.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-01T00:11:35.020 INFO:teuthology.orchestra.run.smithi064.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-01T00:11:35.071 INFO:teuthology.orchestra.run.smithi064.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-01T00:11:35.112 INFO:teuthology.orchestra.run.smithi100.stdout:(30/174): rbd-mirror-19.0.0-1977.g384f31f3.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-01T00:11:35.171 INFO:teuthology.orchestra.run.smithi100.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1977.g384f3 11 MB/s | 3.5 MB 00:00 2024-04-01T00:11:35.254 INFO:teuthology.orchestra.run.smithi100.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1977.g38 2.9 MB/s | 248 kB 00:00 2024-04-01T00:11:35.296 INFO:teuthology.orchestra.run.smithi064.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-04-01T00:11:35.304 INFO:teuthology.orchestra.run.smithi100.stdout:(33/174): ceph-mgr-rook-19.0.0-1977.g384f31f3.e 987 kB/s | 49 kB 00:00 2024-04-01T00:11:35.363 INFO:teuthology.orchestra.run.smithi100.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1977.g3 263 kB/s | 15 kB 00:00 2024-04-01T00:11:35.371 INFO:teuthology.orchestra.run.smithi064.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-01T00:11:35.405 INFO:teuthology.orchestra.run.smithi064.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-01T00:11:35.447 INFO:teuthology.orchestra.run.smithi064.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-01T00:11:35.489 INFO:teuthology.orchestra.run.smithi100.stdout:(35/174): ceph-volume-19.0.0-1977.g384f31f3.el9 2.1 MB/s | 264 kB 00:00 2024-04-01T00:11:35.490 INFO:teuthology.orchestra.run.smithi064.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-01T00:11:35.572 INFO:teuthology.orchestra.run.smithi064.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 856 kB/s | 71 kB 00:00 2024-04-01T00:11:35.614 INFO:teuthology.orchestra.run.smithi064.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 881 kB/s | 36 kB 00:00 2024-04-01T00:11:35.615 INFO:teuthology.orchestra.run.smithi100.stdout:(36/174): cephadm-19.0.0-1977.g384f31f3.el9.noa 5.8 MB/s | 748 kB 00:00 2024-04-01T00:11:35.681 INFO:teuthology.orchestra.run.smithi100.stdout:(37/174): cephfs-top-19.0.0-1977.g384f31f3.el9. 275 kB/s | 18 kB 00:00 2024-04-01T00:11:35.681 INFO:teuthology.orchestra.run.smithi064.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-01T00:11:35.731 INFO:teuthology.orchestra.run.smithi064.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-04-01T00:11:35.774 INFO:teuthology.orchestra.run.smithi064.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 552 kB/s | 23 kB 00:00 2024-04-01T00:11:35.815 INFO:teuthology.orchestra.run.smithi064.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-04-01T00:11:35.823 INFO:teuthology.orchestra.run.smithi100.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-04-01T00:11:35.857 INFO:teuthology.orchestra.run.smithi064.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-04-01T00:11:35.873 INFO:teuthology.orchestra.run.smithi100.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-04-01T00:11:35.898 INFO:teuthology.orchestra.run.smithi100.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 4.1 MB/s | 106 kB 00:00 2024-04-01T00:11:35.899 INFO:teuthology.orchestra.run.smithi064.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-04-01T00:11:35.940 INFO:teuthology.orchestra.run.smithi100.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-04-01T00:11:35.941 INFO:teuthology.orchestra.run.smithi064.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-04-01T00:11:35.983 INFO:teuthology.orchestra.run.smithi064.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 689 kB/s | 29 kB 00:00 2024-04-01T00:11:36.024 INFO:teuthology.orchestra.run.smithi100.stdout:(42/174): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-04-01T00:11:36.049 INFO:teuthology.orchestra.run.smithi100.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 291 kB/s | 32 kB 00:00 2024-04-01T00:11:36.058 INFO:teuthology.orchestra.run.smithi064.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-04-01T00:11:36.124 INFO:teuthology.orchestra.run.smithi100.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-01T00:11:36.133 INFO:teuthology.orchestra.run.smithi064.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-04-01T00:11:36.150 INFO:teuthology.orchestra.run.smithi100.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 175 kB/s | 17 kB 00:00 2024-04-01T00:11:36.175 INFO:teuthology.orchestra.run.smithi100.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-04-01T00:11:36.200 INFO:teuthology.orchestra.run.smithi100.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-01T00:11:36.209 INFO:teuthology.orchestra.run.smithi064.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 30 MB/s | 2.3 MB 00:00 2024-04-01T00:11:36.234 INFO:teuthology.orchestra.run.smithi100.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-04-01T00:11:36.251 INFO:teuthology.orchestra.run.smithi064.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 11 MB/s | 465 kB 00:00 2024-04-01T00:11:36.259 INFO:teuthology.orchestra.run.smithi100.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-04-01T00:11:36.293 INFO:teuthology.orchestra.run.smithi064.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 750 kB/s | 31 kB 00:00 2024-04-01T00:11:36.293 INFO:teuthology.orchestra.run.smithi100.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-01T00:11:36.318 INFO:teuthology.orchestra.run.smithi100.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 802 kB/s | 46 kB 00:00 2024-04-01T00:11:36.344 INFO:teuthology.orchestra.run.smithi100.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-01T00:11:36.369 INFO:teuthology.orchestra.run.smithi100.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-01T00:11:36.376 INFO:teuthology.orchestra.run.smithi064.stdout:(171/174): librados2-19.0.0-1977.g384f31f3.el9. 2.5 MB/s | 3.4 MB 00:01 2024-04-01T00:11:36.394 INFO:teuthology.orchestra.run.smithi100.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-01T00:11:36.419 INFO:teuthology.orchestra.run.smithi100.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-04-01T00:11:36.528 INFO:teuthology.orchestra.run.smithi100.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-04-01T00:11:36.543 INFO:teuthology.orchestra.run.smithi064.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.3 MB/s | 1.1 MB 00:00 2024-04-01T00:11:36.568 INFO:teuthology.orchestra.run.smithi064.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 388 kB/s | 74 kB 00:00 2024-04-01T00:11:36.570 INFO:teuthology.orchestra.run.smithi100.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-01T00:11:36.595 INFO:teuthology.orchestra.run.smithi100.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-01T00:11:36.621 INFO:teuthology.orchestra.run.smithi100.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 132 kB 00:00 2024-04-01T00:11:36.737 INFO:teuthology.orchestra.run.smithi100.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-01T00:11:36.771 INFO:teuthology.orchestra.run.smithi100.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-01T00:11:36.785 INFO:teuthology.orchestra.run.smithi064.stdout:(174/174): librbd1-19.0.0-1977.g384f31f3.el9.x8 1.8 MB/s | 3.1 MB 00:01 2024-04-01T00:11:36.793 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-04-01T00:11:36.793 INFO:teuthology.orchestra.run.smithi064.stdout:Total 17 MB/s | 226 MB 00:13 2024-04-01T00:11:36.796 INFO:teuthology.orchestra.run.smithi100.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-01T00:11:36.930 INFO:teuthology.orchestra.run.smithi100.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-04-01T00:11:36.955 INFO:teuthology.orchestra.run.smithi100.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 156 kB/s | 24 kB 00:00 2024-04-01T00:11:36.981 INFO:teuthology.orchestra.run.smithi100.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-04-01T00:11:37.005 INFO:teuthology.orchestra.run.smithi100.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-01T00:11:37.031 INFO:teuthology.orchestra.run.smithi100.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 629 kB/s | 32 kB 00:00 2024-04-01T00:11:37.156 INFO:teuthology.orchestra.run.smithi100.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-01T00:11:37.172 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-04-01T00:11:37.182 INFO:teuthology.orchestra.run.smithi100.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-01T00:11:37.207 INFO:teuthology.orchestra.run.smithi100.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-01T00:11:37.232 INFO:teuthology.orchestra.run.smithi100.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-01T00:11:37.257 INFO:teuthology.orchestra.run.smithi100.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 819 kB/s | 41 kB 00:00 2024-04-01T00:11:37.283 INFO:teuthology.orchestra.run.smithi100.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-01T00:11:37.308 INFO:teuthology.orchestra.run.smithi100.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-01T00:11:37.325 INFO:teuthology.orchestra.run.smithi100.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-04-01T00:11:37.351 INFO:teuthology.orchestra.run.smithi100.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-01T00:11:37.376 INFO:teuthology.orchestra.run.smithi100.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 846 kB/s | 43 kB 00:00 2024-04-01T00:11:37.469 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-04-01T00:11:37.470 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-04-01T00:11:37.518 INFO:teuthology.orchestra.run.smithi100.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 275 kB/s | 39 kB 00:00 2024-04-01T00:11:37.738 INFO:teuthology.orchestra.run.smithi100.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-04-01T00:11:37.776 INFO:teuthology.orchestra.run.smithi100.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 160 kB/s | 41 kB 00:00 2024-04-01T00:11:37.802 INFO:teuthology.orchestra.run.smithi100.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2024-04-01T00:11:37.819 INFO:teuthology.orchestra.run.smithi100.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 836 kB/s | 35 kB 00:00 2024-04-01T00:11:37.836 INFO:teuthology.orchestra.run.smithi100.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-01T00:11:37.861 INFO:teuthology.orchestra.run.smithi100.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-04-01T00:11:37.878 INFO:teuthology.orchestra.run.smithi100.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-04-01T00:11:37.895 INFO:teuthology.orchestra.run.smithi100.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-01T00:11:37.912 INFO:teuthology.orchestra.run.smithi100.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-01T00:11:37.929 INFO:teuthology.orchestra.run.smithi100.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-04-01T00:11:37.954 INFO:teuthology.orchestra.run.smithi100.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-04-01T00:11:37.987 INFO:teuthology.orchestra.run.smithi100.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-01T00:11:38.013 INFO:teuthology.orchestra.run.smithi100.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-01T00:11:38.038 INFO:teuthology.orchestra.run.smithi100.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-01T00:11:38.064 INFO:teuthology.orchestra.run.smithi100.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-04-01T00:11:38.522 INFO:teuthology.orchestra.run.smithi100.stdout:(94/174): ceph-test-19.0.0-1977.g384f31f3.el9.x 9.9 MB/s | 47 MB 00:04 2024-04-01T00:11:38.697 INFO:teuthology.orchestra.run.smithi100.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 284 kB/s | 187 kB 00:00 2024-04-01T00:11:38.822 INFO:teuthology.orchestra.run.smithi100.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-01T00:11:38.848 INFO:teuthology.orchestra.run.smithi100.stdout:(97/174): libquadmath-11.4.1-3.el9.x86_64.rpm 713 kB/s | 195 kB 00:00 2024-04-01T00:11:38.890 INFO:teuthology.orchestra.run.smithi100.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 972 kB/s | 803 kB 00:00 2024-04-01T00:11:38.940 INFO:teuthology.orchestra.run.smithi100.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-01T00:11:38.966 INFO:teuthology.orchestra.run.smithi100.stdout:(100/174): elfutils-libelf-devel-0.190-2.el9.x8 151 kB/s | 21 kB 00:00 2024-04-01T00:11:39.082 INFO:teuthology.orchestra.run.smithi100.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.7 MB/s | 245 kB 00:00 2024-04-01T00:11:39.107 INFO:teuthology.orchestra.run.smithi100.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.7 MB/s | 248 kB 00:00 2024-04-01T00:11:39.141 INFO:teuthology.orchestra.run.smithi100.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-01T00:11:39.573 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-04-01T00:11:39.573 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-04-01T00:11:39.683 INFO:teuthology.orchestra.run.smithi100.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-04-01T00:11:39.916 INFO:teuthology.orchestra.run.smithi100.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-04-01T00:11:39.967 INFO:teuthology.orchestra.run.smithi100.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-04-01T00:11:40.026 INFO:teuthology.orchestra.run.smithi100.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-04-01T00:11:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:(108/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.2 MB/s | 4.1 MB 00:01 2024-04-01T00:11:40.243 INFO:teuthology.orchestra.run.smithi100.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-01T00:11:40.268 INFO:teuthology.orchestra.run.smithi100.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 292 kB/s | 26 kB 00:00 2024-04-01T00:11:40.293 INFO:teuthology.orchestra.run.smithi100.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-01T00:11:40.319 INFO:teuthology.orchestra.run.smithi100.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-01T00:11:40.344 INFO:teuthology.orchestra.run.smithi100.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-01T00:11:40.378 INFO:teuthology.orchestra.run.smithi100.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-04-01T00:11:40.403 INFO:teuthology.orchestra.run.smithi100.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-04-01T00:11:40.428 INFO:teuthology.orchestra.run.smithi100.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 883 kB/s | 43 kB 00:00 2024-04-01T00:11:40.454 INFO:teuthology.orchestra.run.smithi100.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-04-01T00:11:40.487 INFO:teuthology.orchestra.run.smithi100.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-01T00:11:40.512 INFO:teuthology.orchestra.run.smithi100.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-01T00:11:40.554 INFO:teuthology.orchestra.run.smithi100.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-01T00:11:40.588 INFO:teuthology.orchestra.run.smithi100.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-01T00:11:40.606 INFO:teuthology.orchestra.run.smithi100.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 710 kB/s | 35 kB 00:00 2024-04-01T00:11:40.647 INFO:teuthology.orchestra.run.smithi100.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-01T00:11:40.797 INFO:teuthology.orchestra.run.smithi100.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-01T00:11:40.914 INFO:teuthology.orchestra.run.smithi100.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 546 kB/s | 144 kB 00:00 2024-04-01T00:11:40.939 INFO:teuthology.orchestra.run.smithi100.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-01T00:11:40.981 INFO:teuthology.orchestra.run.smithi100.stdout:(127/174): protobuf-compiler-3.14.0-13.el9.x86_ 469 kB/s | 863 kB 00:01 2024-04-01T00:11:41.006 INFO:teuthology.orchestra.run.smithi100.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-01T00:11:41.032 INFO:teuthology.orchestra.run.smithi100.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-04-01T00:11:41.057 INFO:teuthology.orchestra.run.smithi100.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-01T00:11:41.090 INFO:teuthology.orchestra.run.smithi100.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-01T00:11:41.124 INFO:teuthology.orchestra.run.smithi100.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-01T00:11:41.149 INFO:teuthology.orchestra.run.smithi100.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-01T00:11:41.174 INFO:teuthology.orchestra.run.smithi100.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 961 kB/s | 79 kB 00:00 2024-04-01T00:11:41.200 INFO:teuthology.orchestra.run.smithi100.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2024-04-01T00:11:41.225 INFO:teuthology.orchestra.run.smithi100.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-01T00:11:41.250 INFO:teuthology.orchestra.run.smithi100.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-01T00:11:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-01T00:11:41.301 INFO:teuthology.orchestra.run.smithi100.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-01T00:11:41.326 INFO:teuthology.orchestra.run.smithi100.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-01T00:11:41.352 INFO:teuthology.orchestra.run.smithi100.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-01T00:11:41.377 INFO:teuthology.orchestra.run.smithi100.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-04-01T00:11:41.402 INFO:teuthology.orchestra.run.smithi100.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-01T00:11:41.436 INFO:teuthology.orchestra.run.smithi100.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-04-01T00:11:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-01T00:11:41.486 INFO:teuthology.orchestra.run.smithi100.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-01T00:11:41.511 INFO:teuthology.orchestra.run.smithi100.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-01T00:11:41.537 INFO:teuthology.orchestra.run.smithi100.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-01T00:11:41.562 INFO:teuthology.orchestra.run.smithi100.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-01T00:11:41.604 INFO:teuthology.orchestra.run.smithi100.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-01T00:11:41.738 INFO:teuthology.orchestra.run.smithi100.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-04-01T00:11:41.855 INFO:teuthology.orchestra.run.smithi100.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-04-01T00:11:41.888 INFO:teuthology.orchestra.run.smithi100.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-01T00:11:41.930 INFO:teuthology.orchestra.run.smithi100.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-01T00:11:41.980 INFO:teuthology.orchestra.run.smithi100.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-01T00:11:42.039 INFO:teuthology.orchestra.run.smithi100.stdout:(156/174): librbd1-19.0.0-1977.g384f31f3.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-04-01T00:11:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 534 kB/s | 71 kB 00:00 2024-04-01T00:11:42.139 INFO:teuthology.orchestra.run.smithi100.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-04-01T00:11:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:(159/174): librados2-19.0.0-1977.g384f31f3.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-01T00:11:42.231 INFO:teuthology.orchestra.run.smithi100.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-01T00:11:42.265 INFO:teuthology.orchestra.run.smithi100.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-04-01T00:11:42.290 INFO:teuthology.orchestra.run.smithi100.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.7 MB/s | 256 kB 00:00 2024-04-01T00:11:42.332 INFO:teuthology.orchestra.run.smithi100.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-01T00:11:42.358 INFO:teuthology.orchestra.run.smithi100.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 862 kB/s | 108 kB 00:00 2024-04-01T00:11:42.383 INFO:teuthology.orchestra.run.smithi100.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-04-01T00:11:42.400 INFO:teuthology.orchestra.run.smithi100.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 950 kB/s | 64 kB 00:00 2024-04-01T00:11:42.416 INFO:teuthology.orchestra.run.smithi100.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 490 kB/s | 29 kB 00:00 2024-04-01T00:11:42.500 INFO:teuthology.orchestra.run.smithi100.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-01T00:11:42.559 INFO:teuthology.orchestra.run.smithi100.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-04-01T00:11:42.609 INFO:teuthology.orchestra.run.smithi100.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-04-01T00:11:42.634 INFO:teuthology.orchestra.run.smithi100.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-04-01T00:11:42.701 INFO:teuthology.orchestra.run.smithi100.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-04-01T00:11:42.754 INFO:teuthology.orchestra.run.smithi100.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 636 kB/s | 74 kB 00:00 2024-04-01T00:11:42.793 INFO:teuthology.orchestra.run.smithi100.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.1 MB/s | 2.3 MB 00:00 2024-04-01T00:11:42.801 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-04-01T00:11:42.802 INFO:teuthology.orchestra.run.smithi100.stdout:Total 18 MB/s | 226 MB 00:12 2024-04-01T00:11:43.172 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-04-01T00:11:43.445 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-04-01T00:11:43.445 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-04-01T00:11:45.463 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-04-01T00:11:45.463 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-04-01T00:11:46.591 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-04-01T00:11:46.707 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-01T00:11:46.733 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-01T00:11:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-01T00:11:46.772 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-01T00:11:46.813 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-01T00:11:46.867 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-01T00:11:46.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-01T00:11:46.941 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-01T00:11:46.986 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-01T00:11:47.328 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-01T00:11:47.379 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:11:47.487 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:11:47.513 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:11:47.588 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:11:47.618 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 12/198 2024-04-01T00:11:47.659 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-01T00:11:47.863 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-01T00:11:47.907 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-01T00:11:47.935 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:11:48.023 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:11:48.044 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 17/198 2024-04-01T00:11:48.109 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 17/198 2024-04-01T00:11:48.148 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-01T00:11:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-01T00:11:48.186 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-01T00:11:48.329 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-01T00:11:48.376 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-01T00:11:48.426 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-01T00:11:48.759 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-01T00:11:48.795 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-01T00:11:48.855 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-01T00:11:48.968 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:49.015 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:49.248 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:49.356 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-01T00:11:49.460 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-01T00:11:49.503 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-01T00:11:49.530 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-01T00:11:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-01T00:11:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-01T00:11:49.711 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-01T00:11:49.748 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-01T00:11:49.815 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-01T00:11:49.848 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-01T00:11:49.891 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-01T00:11:49.924 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-01T00:11:49.951 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-01T00:11:50.023 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-01T00:11:50.057 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-1977.g384f31f3.el9. 39/198 2024-04-01T00:11:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el 40/198 2024-04-01T00:11:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_6 41/198 2024-04-01T00:11:50.192 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-01T00:11:50.234 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-01T00:11:50.311 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-01T00:11:50.451 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-01T00:11:50.504 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-01T00:11:51.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-01T00:11:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-01T00:11:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-01T00:11:51.277 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-01T00:11:51.286 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-01T00:11:51.332 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-01T00:11:51.375 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-01T00:11:51.404 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_6 53/198 2024-04-01T00:11:51.434 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-01T00:11:51.708 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-01T00:11:51.904 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-01T00:11:51.994 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-01T00:11:52.039 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-01T00:11:52.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-01T00:11:52.227 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-01T00:11:52.284 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-01T00:11:52.440 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-04-01T00:11:52.540 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-01T00:11:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-01T00:11:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-01T00:11:52.638 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-01T00:11:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-01T00:11:52.670 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-01T00:11:52.687 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-01T00:11:52.707 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-01T00:11:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-01T00:11:52.756 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-01T00:11:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-01T00:11:52.790 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-01T00:11:52.807 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-01T00:11:52.833 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-01T00:11:52.858 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-01T00:11:52.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-01T00:11:53.042 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-01T00:11:53.096 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-01T00:11:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-01T00:11:53.168 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-01T00:11:53.184 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-01T00:11:53.237 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:11:53.267 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-01T00:11:53.310 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-01T00:11:53.349 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-01T00:11:53.368 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:11:53.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-01T00:11:53.397 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:11:53.399 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-01T00:11:53.478 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-01T00:11:53.506 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:11:53.541 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 12/198 2024-04-01T00:11:53.586 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-01T00:11:53.604 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-01T00:11:53.712 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-01T00:11:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-01T00:11:53.769 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-01T00:11:53.808 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-01T00:11:53.832 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-01T00:11:53.860 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:11:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-01T00:11:53.947 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:11:53.978 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 17/198 2024-04-01T00:11:54.043 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 17/198 2024-04-01T00:11:54.062 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-01T00:11:54.082 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-01T00:11:54.104 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-01T00:11:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-01T00:11:54.120 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-01T00:11:54.255 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-01T00:11:54.260 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-01T00:11:54.293 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-01T00:11:54.298 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-01T00:11:54.325 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-01T00:11:54.333 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-01T00:11:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-01T00:11:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-01T00:11:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-01T00:11:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-01T00:11:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-01T00:11:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-01T00:11:54.546 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-01T00:11:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-01T00:11:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-01T00:11:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-01T00:11:54.686 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-01T00:11:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-01T00:11:54.730 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-01T00:11:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-01T00:11:54.830 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-01T00:11:54.890 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-01T00:11:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-01T00:11:55.026 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-01T00:11:55.075 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-01T00:11:55.107 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-01T00:11:55.177 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-01T00:11:55.228 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-01T00:11:55.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-01T00:11:55.322 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-01T00:11:55.353 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-01T00:11:55.380 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-01T00:11:55.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-01T00:11:55.471 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-01T00:11:55.526 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-01T00:11:55.565 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-01T00:11:55.628 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-01T00:11:55.664 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-01T00:11:55.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-01T00:11:55.740 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-01T00:11:55.768 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-01T00:11:55.837 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-01T00:11:55.873 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-1977.g384f31f3.el9. 39/198 2024-04-01T00:11:55.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el 40/198 2024-04-01T00:11:55.945 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_6 41/198 2024-04-01T00:11:56.004 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-01T00:11:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-01T00:11:56.050 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-01T00:11:56.059 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-01T00:11:56.083 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-01T00:11:56.132 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-01T00:11:56.143 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-01T00:11:56.235 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-01T00:11:56.268 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-01T00:11:56.278 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-01T00:11:56.311 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-01T00:11:56.460 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-01T00:11:56.491 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-01T00:11:56.578 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-01T00:11:56.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-01T00:11:56.915 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-01T00:11:56.994 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-01T00:11:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-01T00:11:57.045 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-01T00:11:57.079 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-01T00:11:57.132 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-01T00:11:57.170 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_6 53/198 2024-04-01T00:11:57.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-01T00:11:57.270 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-01T00:11:57.386 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-01T00:11:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-01T00:11:57.756 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-01T00:11:57.863 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-01T00:11:57.907 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-01T00:11:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-01T00:11:58.189 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-01T00:11:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-01T00:11:58.516 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-01T00:11:58.553 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-01T00:11:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-01T00:11:58.637 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-01T00:11:58.672 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-01T00:11:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-01T00:11:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-01T00:11:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-01T00:11:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-01T00:11:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-01T00:11:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-01T00:11:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-01T00:11:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-01T00:11:58.946 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-01T00:11:58.976 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-01T00:11:59.017 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-01T00:11:59.104 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-01T00:11:59.167 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-01T00:11:59.207 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-01T00:11:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-01T00:11:59.233 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-01T00:11:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 124/198 2024-04-01T00:11:59.258 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-01T00:11:59.328 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 124/198 2024-04-01T00:11:59.338 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-01T00:11:59.358 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 125/198 2024-04-01T00:11:59.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-01T00:11:59.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-01T00:11:59.567 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-01T00:11:59.617 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-01T00:11:59.674 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-01T00:11:59.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-01T00:11:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-01T00:11:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 128/198 2024-04-01T00:11:59.922 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-01T00:11:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 128/198 2024-04-01T00:11:59.971 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-01T00:12:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 129/198 2024-04-01T00:12:00.107 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-01T00:12:00.152 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-01T00:12:00.184 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-01T00:12:00.215 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-01T00:12:00.244 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-01T00:12:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-01T00:12:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-01T00:12:00.348 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-01T00:12:00.378 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-01T00:12:00.446 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-01T00:12:00.480 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-01T00:12:00.514 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-01T00:12:00.555 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-01T00:12:00.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-01T00:12:00.679 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-01T00:12:00.769 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-01T00:12:00.821 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-01T00:12:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-01T00:12:00.899 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-01T00:12:00.996 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-01T00:12:01.045 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-01T00:12:01.766 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-01T00:12:01.826 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-01T00:12:01.859 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-01T00:12:01.900 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-01T00:12:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-01T00:12:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-01T00:12:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-01T00:12:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-01T00:12:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-01T00:12:02.341 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:02.371 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:02.953 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-01T00:12:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 131/198 2024-04-01T00:12:03.065 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-01T00:12:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 131/198 2024-04-01T00:12:03.082 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-01T00:12:03.082 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-01T00:12:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 132/198 2024-04-01T00:12:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-01T00:12:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-01T00:12:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-01T00:12:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-01T00:12:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-01T00:12:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-01T00:12:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 124/198 2024-04-01T00:12:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 124/198 2024-04-01T00:12:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 125/198 2024-04-01T00:12:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-01T00:12:05.421 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-01T00:12:05.642 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 128/198 2024-04-01T00:12:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 128/198 2024-04-01T00:12:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 129/198 2024-04-01T00:12:08.125 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:08.157 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:08.702 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 130/198 2024-04-01T00:12:08.735 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 131/198 2024-04-01T00:12:08.814 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 131/198 2024-04-01T00:12:08.814 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-01T00:12:08.814 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-01T00:12:08.814 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:08.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 132/198 2024-04-01T00:12:14.870 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 132/198 2024-04-01T00:12:14.870 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-04-01T00:12:14.870 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-04-01T00:12:14.870 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-04-01T00:12:14.870 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-04-01T00:12:14.871 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-04-01T00:12:14.871 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-04-01T00:12:14.871 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-04-01T00:12:14.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:15.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 133/198 2024-04-01T00:12:15.053 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 133/198 2024-04-01T00:12:15.053 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:15.053 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-01T00:12:15.053 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:16.601 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 134/198 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 134/198 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-01T00:12:16.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:16.859 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 135/198 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 135/198 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-01T00:12:16.882 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:17.352 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 136/198 2024-04-01T00:12:17.374 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 136/198 2024-04-01T00:12:17.374 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:17.374 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-01T00:12:17.374 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-01T00:12:17.374 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-01T00:12:17.375 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:17.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-01T00:12:17.929 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 138/198 2024-04-01T00:12:17.961 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 138/198 2024-04-01T00:12:17.992 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.e 139/198 2024-04-01T00:12:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3. 140/198 2024-04-01T00:12:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 141/198 2024-04-01T00:12:19.165 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 141/198 2024-04-01T00:12:19.201 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 142/198 2024-04-01T00:12:20.214 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 142/198 2024-04-01T00:12:20.235 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 143/198 2024-04-01T00:12:20.341 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 143/198 2024-04-01T00:12:20.513 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el 144/198 2024-04-01T00:12:20.538 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 145/198 2024-04-01T00:12:20.560 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 145/198 2024-04-01T00:12:20.560 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:20.561 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-01T00:12:20.561 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-01T00:12:20.561 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-01T00:12:20.561 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:20.605 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 146/198 2024-04-01T00:12:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 146/198 2024-04-01T00:12:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 132/198 2024-04-01T00:12:20.632 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-04-01T00:12:20.632 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-04-01T00:12:20.632 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-04-01T00:12:20.633 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-04-01T00:12:20.633 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-04-01T00:12:20.633 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-04-01T00:12:20.633 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-04-01T00:12:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:20.670 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 147/198 2024-04-01T00:12:20.799 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 133/198 2024-04-01T00:12:20.814 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 133/198 2024-04-01T00:12:20.814 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:20.814 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-01T00:12:20.814 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-01T00:12:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 149/198 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 149/198 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-01T00:12:21.622 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:21.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1977.g384f3 150/198 2024-04-01T00:12:21.696 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1977.g384f3 150/198 2024-04-01T00:12:21.696 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:21.696 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-01T00:12:21.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:21.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:21.763 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:21.764 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:21.764 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-01T00:12:21.764 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-01T00:12:21.764 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-01T00:12:21.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 152/198 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 152/198 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-01T00:12:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:22.313 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 134/198 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 134/198 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-01T00:12:22.338 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:22.571 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 135/198 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 135/198 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-01T00:12:22.594 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:23.030 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 136/198 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 136/198 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-01T00:12:23.053 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:23.140 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-01T00:12:23.608 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 138/198 2024-04-01T00:12:23.639 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 138/198 2024-04-01T00:12:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.e 139/198 2024-04-01T00:12:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3. 140/198 2024-04-01T00:12:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 141/198 2024-04-01T00:12:24.795 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 141/198 2024-04-01T00:12:24.830 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 142/198 2024-04-01T00:12:25.791 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 142/198 2024-04-01T00:12:25.814 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 143/198 2024-04-01T00:12:25.913 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 143/198 2024-04-01T00:12:26.079 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el 144/198 2024-04-01T00:12:26.100 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 145/198 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 145/198 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-01T00:12:26.123 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:26.175 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 146/198 2024-04-01T00:12:26.186 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 146/198 2024-04-01T00:12:26.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 147/198 2024-04-01T00:12:26.780 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 153/198 2024-04-01T00:12:26.822 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 154/198 2024-04-01T00:12:26.860 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 155/198 2024-04-01T00:12:26.888 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-01T00:12:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 157/198 2024-04-01T00:12:27.101 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-01T00:12:27.127 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 149/198 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 149/198 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-01T00:12:27.150 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:27.190 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-01T00:12:27.203 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1977.g384f3 150/198 2024-04-01T00:12:27.225 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1977.g384f3 150/198 2024-04-01T00:12:27.225 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:27.225 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-01T00:12:27.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:27.244 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 159/198 2024-04-01T00:12:27.270 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-01T00:12:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:27.301 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-01T00:12:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-01T00:12:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-01T00:12:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_ 163/198 2024-04-01T00:12:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-01T00:12:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-01T00:12:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-01T00:12:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 152/198 2024-04-01T00:12:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 152/198 2024-04-01T00:12:27.618 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-01T00:12:27.619 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-01T00:12:27.619 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-01T00:12:27.619 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-01T00:12:27.619 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-01T00:12:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-01T00:12:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:29.167 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 170/198 2024-04-01T00:12:29.256 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-01T00:12:29.298 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-01T00:12:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-01T00:12:29.427 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-01T00:12:29.452 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-01T00:12:29.477 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-01T00:12:29.504 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-01T00:12:29.504 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-01T00:12:29.583 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-01T00:12:29.621 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-01T00:12:29.621 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-01T00:12:29.689 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-01T00:12:29.716 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-01T00:12:29.748 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-01T00:12:29.774 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-01T00:12:29.799 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-01T00:12:29.833 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-01T00:12:29.886 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-01T00:12:29.920 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-01T00:12:29.945 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-01T00:12:29.970 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-01T00:12:29.996 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-01T00:12:30.021 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-01T00:12:30.028 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:30.028 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:30.216 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:30.216 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-01T00:12:30.290 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-01T00:12:30.322 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-01T00:12:30.349 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-01T00:12:30.375 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-01T00:12:30.400 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-01T00:12:30.468 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-01T00:12:32.050 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 153/198 2024-04-01T00:12:32.092 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 154/198 2024-04-01T00:12:32.122 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 155/198 2024-04-01T00:12:32.149 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-01T00:12:32.334 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 157/198 2024-04-01T00:12:32.450 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-01T00:12:32.489 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 159/198 2024-04-01T00:12:32.542 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-01T00:12:32.598 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-01T00:12:32.641 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-01T00:12:32.673 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_ 163/198 2024-04-01T00:12:32.705 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-01T00:12:32.735 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-01T00:12:32.807 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-01T00:12:32.875 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-01T00:12:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-01T00:12:33.491 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:33.532 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:33.552 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-01T00:12:34.372 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 170/198 2024-04-01T00:12:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-01T00:12:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-01T00:12:34.559 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-01T00:12:34.597 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-01T00:12:34.622 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-01T00:12:34.648 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-01T00:12:34.674 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-01T00:12:34.675 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-01T00:12:34.770 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-01T00:12:34.800 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-01T00:12:34.800 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-01T00:12:34.887 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-01T00:12:34.913 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-01T00:12:34.946 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-01T00:12:34.972 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-01T00:12:34.996 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-01T00:12:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-01T00:12:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-01T00:12:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-01T00:12:35.126 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-01T00:12:35.152 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-01T00:12:35.177 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-01T00:12:35.202 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-01T00:12:35.209 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:35.209 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:35.389 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-01T00:12:35.389 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-01T00:12:35.472 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-01T00:12:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-01T00:12:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-01T00:12:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-01T00:12:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-01T00:12:35.647 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-01T00:12:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-01T00:12:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 1/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 2/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 3/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 4/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1977.g384f3 5/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 6/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 7/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 8/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 9/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 12/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 13/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 14/198 2024-04-01T00:12:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_ 15/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 17/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_6 18/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 19/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 20/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el 21/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-1977.g384f31f3.el9. 22/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_6 23/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 24/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 25/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 26/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 27/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 28/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 29/198 2024-04-01T00:12:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3. 30/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 31/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 32/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 33/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el 34/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 35/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.e 36/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 37/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 38/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 39/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-01T00:12:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-01T00:12:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-01T00:12:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-01T00:12:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-01T00:12:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-01T00:12:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-01T00:12:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-01T00:12:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 153/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-01T00:12:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-01T00:12:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-01T00:12:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-01T00:12:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: acl-2.3.1-4.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-01T00:12:39.079 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.080 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.081 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-16.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-01T00:12:39.083 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-01T00:12:39.084 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.14-1.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-01T00:12:39.085 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-01T00:12:39.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-01T00:12:39.087 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-01T00:12:39.088 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.089 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-01T00:12:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-01T00:12:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-01T00:12:39.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:39.092 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-04-01T00:12:39.544 DEBUG:teuthology.parallel:result is None 2024-04-01T00:12:41.064 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-01T00:12:41.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 1/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 2/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 3/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 4/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1977.g384f3 5/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 6/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 7/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 8/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 9/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 10/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 11/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 12/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 13/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 14/198 2024-04-01T00:12:41.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_ 15/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 16/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 17/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_6 18/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86 19/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 20/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el 21/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-1977.g384f31f3.el9. 22/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_6 23/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 24/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 25/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 26/198 2024-04-01T00:12:41.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 27/198 2024-04-01T00:12:41.081 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 28/198 2024-04-01T00:12:41.081 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 29/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3. 30/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noa 31/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.n 32/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1977.g384 33/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el 34/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 35/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.e 36/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 37/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 38/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 39/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-01T00:12:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-01T00:12:41.083 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-01T00:12:41.084 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-01T00:12:41.085 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-01T00:12:41.086 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-01T00:12:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-01T00:12:41.088 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-01T00:12:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 151/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 153/198 2024-04-01T00:12:41.090 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-01T00:12:41.091 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-01T00:12:41.092 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-01T00:12:41.093 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-01T00:12:44.061 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.062 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.063 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.064 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-1977.g384f31f3.el9.noarch 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-01T00:12:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-01T00:12:44.066 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-04-01T00:12:44.067 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.14-1.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-01T00:12:44.068 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-01T00:12:44.069 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-01T00:12:44.070 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-01T00:12:44.071 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-01T00:12:44.072 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-01T00:12:44.073 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-1977.g384f31f3.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-01T00:12:44.074 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-01T00:12:44.075 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-01T00:12:44.075 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:44.075 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-04-01T00:12:44.542 DEBUG:teuthology.parallel:result is None 2024-04-01T00:12:44.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384f31f3d3aee7f8ba3f60846b8844c049165583 2024-04-01T00:12:44.761 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-01T00:12:44.786 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-1977.g384f31f3.el9 2024-04-01T00:12:44.787 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1977.g384f31f3.el9 2024-04-01T00:12:44.787 INFO:teuthology.task.install:The correct ceph version 19.0.0-1977.g384f31f3 is installed. 2024-04-01T00:12:44.789 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384f31f3d3aee7f8ba3f60846b8844c049165583 2024-04-01T00:12:44.934 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-01T00:12:44.957 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1977.g384f31f3.el9 2024-04-01T00:12:44.957 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1977.g384f31f3.el9 2024-04-01T00:12:44.957 INFO:teuthology.task.install:The correct ceph version 19.0.0-1977.g384f31f3 is installed. 2024-04-01T00:12:44.959 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-01T00:12:44.960 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:44.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-01T00:12:44.995 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:44.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-01T00:12:45.028 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-01T00:12:45.029 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:45.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-04-01T00:12:45.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-01T00:12:45.136 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:45.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-04-01T00:12:45.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-01T00:12:45.234 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-01T00:12:45.234 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:45.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-01T00:12:45.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-01T00:12:45.336 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:45.337 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-01T00:12:45.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-01T00:12:45.432 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-01T00:12:45.433 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:45.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-04-01T00:12:45.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-01T00:12:45.533 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:45.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-04-01T00:12:45.561 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-01T00:12:45.629 INFO:teuthology.run_tasks:Running task ceph... 2024-04-01T00:12:45.712 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-01T00:12:45.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2024-04-01T00:12:45.715 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-04-01T00:12:45.742 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-01T00:12:45.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-01T00:12:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-01T00:12:45.812 INFO:tasks.ceph:Creating extra log directories... 2024-04-01T00:12:45.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-01T00:12:45.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-01T00:12:45.891 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-01T00:12:45.892 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-01T00:12:45.892 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-01T00:12:45.892 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:45.892 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-01T00:12:45.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-01T00:12:46.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-01T00:12:46.087 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:46.087 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-01T00:12:46.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-01T00:12:46.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-01T00:12:46.254 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-01T00:12:46.254 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-01T00:12:46.254 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7633198', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8238, 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583', 'sleep_before_teardown': 0, 'subset': '361/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO47IA+EvSR0YgNt5WX/u0uLqP/LctJOxCRWkyqMGKRxSM5C6a6eiCyA2wcSO6kp0CNMI7Qqft9C11I/X8lVYQ=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoLc4FeywrK/SqV54UHq0a0Sj8ZFU63fFNKCIdmYT78xPktjt7ncDU1VQRYkEOPbhv+QsYrsbm8EXPM5SvTa0I='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '384f31f3d3aee7f8ba3f60846b8844c049165583', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-31_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-01T00:12:46.255 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-01T00:12:46.257 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-01T00:12:46.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2024-04-01T00:12:46.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-04-01T00:12:46.370 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:46.370 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-04-01T00:12:46.391 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-01T00:12:46.391 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-01 00:12:30.820802608 +0000 2024-04-01T00:12:46.446 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-01 00:10:32.160391663 +0000 2024-04-01T00:12:46.447 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-01 00:10:32.160391663 +0000 2024-04-01T00:12:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-01 00:10:32.160391663 +0000 2024-04-01T00:12:46.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-01T00:12:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-01T00:12:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-01T00:12:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000250196 s, 2.0 MB/s 2024-04-01T00:12:46.513 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-01T00:12:46.569 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:46.623 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-01 00:12:30.820802608 +0000 2024-04-01T00:12:46.624 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-01 00:10:32.494384996 +0000 2024-04-01T00:12:46.624 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-01 00:10:32.494384996 +0000 2024-04-01T00:12:46.624 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-01 00:10:32.494384996 +0000 2024-04-01T00:12:46.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-01T00:12:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-01T00:12:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-01T00:12:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000180859 s, 2.8 MB/s 2024-04-01T00:12:46.691 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-01T00:12:46.746 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-01 00:12:30.821802589 +0000 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-01 00:10:32.835378189 +0000 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-01 00:10:32.835378189 +0000 2024-04-01T00:12:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-01 00:10:32.835378189 +0000 2024-04-01T00:12:46.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-01T00:12:46.864 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-01T00:12:46.864 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-01T00:12:46.864 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000197869 s, 2.6 MB/s 2024-04-01T00:12:46.865 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-01T00:12:46.921 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-01 00:12:30.821802589 +0000 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-01 00:10:33.165371602 +0000 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-01 00:10:33.165371602 +0000 2024-04-01T00:12:46.975 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-01 00:10:33.165371602 +0000 2024-04-01T00:12:46.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-01T00:12:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-01T00:12:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-01T00:12:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000159 s, 3.2 MB/s 2024-04-01T00:12:47.042 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-01T00:12:47.097 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:47.098 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:47.098 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-04-01T00:12:47.114 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-01T00:12:47.115 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-01 00:12:35.982647869 +0000 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-01 00:10:28.621864540 +0000 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-01 00:10:28.621864540 +0000 2024-04-01T00:12:47.171 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-01 00:10:28.621864540 +0000 2024-04-01T00:12:47.172 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-01T00:12:47.238 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-01T00:12:47.238 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-01T00:12:47.238 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000277383 s, 1.8 MB/s 2024-04-01T00:12:47.240 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-01T00:12:47.296 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-01 00:12:35.982647869 +0000 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-01 00:10:28.963856057 +0000 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-01 00:10:28.963856057 +0000 2024-04-01T00:12:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-01 00:10:28.963856057 +0000 2024-04-01T00:12:47.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-01T00:12:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-01T00:12:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-01T00:12:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000194227 s, 2.6 MB/s 2024-04-01T00:12:47.420 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-01T00:12:47.477 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-01 00:12:35.983647845 +0000 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-01 00:10:29.318847251 +0000 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-01 00:10:29.318847251 +0000 2024-04-01T00:12:47.534 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-01 00:10:29.318847251 +0000 2024-04-01T00:12:47.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-01T00:12:47.599 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-01T00:12:47.599 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-01T00:12:47.599 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000220208 s, 2.3 MB/s 2024-04-01T00:12:47.600 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-01T00:12:47.656 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-04-01T00:12:47.711 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-01T00:12:47.711 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-01T00:12:47.711 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-01T00:12:47.711 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-01T00:12:47.712 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-01T00:12:47.712 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-01 00:12:35.983647845 +0000 2024-04-01T00:12:47.712 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-01 00:10:29.653838941 +0000 2024-04-01T00:12:47.712 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-01 00:10:29.653838941 +0000 2024-04-01T00:12:47.712 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-01 00:10:29.653838941 +0000 2024-04-01T00:12:47.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-01T00:12:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-01T00:12:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-01T00:12:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000227514 s, 2.3 MB/s 2024-04-01T00:12:47.778 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-01T00:12:47.834 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:47.834 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-01T00:12:47.834 INFO:tasks.ceph:Generating config... 2024-04-01T00:12:47.835 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-01T00:12:47.835 INFO:tasks.ceph:[client] debug client = 20 2024-04-01T00:12:47.836 INFO:tasks.ceph:[client] debug ms = 1 2024-04-01T00:12:47.836 INFO:tasks.ceph:[client] pid file = 2024-04-01T00:12:47.836 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-01T00:12:47.836 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-01T00:12:47.836 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-01T00:12:47.837 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-01T00:12:47.837 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-01T00:12:47.838 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-01T00:12:47.838 INFO:tasks.ceph:Setting up mon.a... 2024-04-01T00:12:47.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-01T00:12:47.888 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2024-04-01T00:12:47.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-01T00:12:47.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-01T00:12:47.964 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2024-04-01T00:12:47.965 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-01T00:12:47.965 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:47.965 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-01T00:12:48.019 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:48.105 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid 05d5f08b-c3c0-4c90-b927-5e55cbe6aed5 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 05d5f08b-c3c0-4c90-b927-5e55cbe6aed5 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2024-04-01T00:12:48.105948+0000 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-04-01T00:12:48.105948+0000 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 17 (quincy) 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-01T00:12:48.106 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-01T00:12:48.107 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-01T00:12:48.122 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 05d5f08b-c3c0-4c90-b927-5e55cbe6aed5... 2024-04-01T00:12:48.123 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-01T00:12:48.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.213 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.214 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.215 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 05d5f08b-c3c0-4c90-b927-5e55cbe6aed5 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.216 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2024-04-01T00:12:48.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2024-04-01T00:12:48.218 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-04-01T00:12:48.219 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2024-04-01T00:12:48.220 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-01T00:12:48.221 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: mon down mkfs grace = 300 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-01T00:12:48.222 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2024-04-01T00:12:48.223 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-04-01T00:12:48.224 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-01T00:12:48.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-01T00:12:48.226 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-01T00:12:48.227 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.64,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 05d5f08b-c3c0-4c90-b927-5e55cbe6aed5 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-01T00:12:48.228 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-01T00:12:48.229 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-01T00:12:48.230 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-01T00:12:48.231 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-01T00:12:48.232 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon down mkfs grace = 300 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-01T00:12:48.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-01T00:12:48.234 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-01T00:12:48.235 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-01T00:12:48.236 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-01T00:12:48.237 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-01T00:12:48.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-01T00:12:48.332 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-01T00:12:48.333 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:48.333 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-01T00:12:48.347 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:48.347 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-01T00:12:48.402 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:12:48.402 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:48.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-01T00:12:48.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-01T00:12:48.481 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:12:48.481 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:48.536 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:12:48.536 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:48.536 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-01T00:12:48.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-01T00:12:48.577 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:12:48.577 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:48.633 INFO:tasks.ceph:Setting up mon nodes... 2024-04-01T00:12:48.633 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-01T00:12:48.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-01T00:12:48.688 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-01T00:12:48.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-01T00:12:48.754 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-01T00:12:48.757 INFO:tasks.ceph:Setting up mds nodes... 2024-04-01T00:12:48.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-01T00:12:48.811 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-01T00:12:48.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-01T00:12:48.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-01T00:12:48.933 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-01T00:12:48.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-01T00:12:48.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-01T00:12:49.016 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-01T00:12:49.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-01T00:12:49.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-01T00:12:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-01T00:12:49.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-01T00:12:49.171 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-01T00:12:49.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-01T00:12:49.213 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-01T00:12:49.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-01T00:12:49.310 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-01T00:12:49.325 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-01T00:12:49.367 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-01T00:12:49.383 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-01T00:12:49.464 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-01T00:12:49.479 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-01T00:12:49.479 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-01T00:12:49.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-01T00:12:49.506 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:49.506 INFO:tasks.ceph:role: osd.0 2024-04-01T00:12:49.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:12:49.507 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-01T00:12:49.580 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:12:49.581 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:12:50.080 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-04-01T00:12:50.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-04-01T00:12:50.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-01T00:12:50.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-01T00:12:50.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-01T00:12:50.292 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:50.292 INFO:tasks.ceph:role: osd.1 2024-04-01T00:12:50.292 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:12:50.293 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:12:50.367 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:12:50.368 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:12:50.368 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:12:50.368 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:12:50.827 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-04-01T00:12:50.830 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-04-01T00:12:50.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-01T00:12:50.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-01T00:12:50.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-01T00:12:51.043 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:51.043 INFO:tasks.ceph:role: osd.2 2024-04-01T00:12:51.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:12:51.043 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-01T00:12:51.117 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:12:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:12:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:12:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:12:51.117 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:12:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:12:51.118 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:12:51.118 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:12:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:12:51.118 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:12:51.599 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-04-01T00:12:51.601 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-04-01T00:12:51.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-01T00:12:51.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-01T00:12:51.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-01T00:12:51.794 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-01T00:12:51.794 INFO:tasks.ceph:role: osd.3 2024-04-01T00:12:51.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:12:51.794 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-01T00:12:51.868 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:12:51.868 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:12:51.868 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:12:51.868 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:12:51.869 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:12:52.385 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-04-01T00:12:52.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-04-01T00:12:52.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-01T00:12:52.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-01T00:12:52.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.717+0000 7f6866c0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-01T00:12:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.721+0000 7f6866c0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-01T00:12:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.721+0000 7f6866c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-01T00:12:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.721+0000 7f6866c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-01T00:12:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.721+0000 7f6866c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-01T00:12:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:52.728+0000 7f6866c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-01T00:12:55.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-01T00:12:55.621 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.712+0000 7f03a7c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.713+0000 7f03a7c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.713+0000 7f03a7c15640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.713+0000 7f03a7c15640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.713+0000 7f03a7c15640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-01T00:12:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:55.713+0000 7f03a7c15640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-01T00:12:58.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-01T00:12:58.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:12:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.683+0000 7fdd6c70d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-01T00:12:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.683+0000 7fdd6c70d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-01T00:12:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.683+0000 7fdd6c70d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-01T00:12:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.683+0000 7fdd6c70d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-01T00:12:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.683+0000 7fdd6c70d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-01T00:12:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:12:58.684+0000 7fdd6c70d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-01T00:13:01.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-01T00:13:01.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.659+0000 7f553eb8c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-01T00:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.659+0000 7f553eb8c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-01T00:13:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.659+0000 7f553eb8c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-01T00:13:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.659+0000 7f553eb8c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-01T00:13:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.660+0000 7f553eb8c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-01T00:13:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:01.660+0000 7f553eb8c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-01T00:13:04.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-01T00:13:04.551 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-01T00:13:04.583 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-01T00:13:04.584 INFO:tasks.ceph:role: osd.4 2024-04-01T00:13:04.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:13:04.584 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:13:04.661 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:13:04.662 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:13:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:13:04.662 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:13:05.163 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-01T00:13:05.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-01T00:13:05.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-01T00:13:05.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-01T00:13:05.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-01T00:13:05.373 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-01T00:13:05.374 INFO:tasks.ceph:role: osd.5 2024-04-01T00:13:05.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:13:05.374 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-01T00:13:05.447 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:13:05.447 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:13:05.448 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:13:05.895 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-01T00:13:05.897 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-01T00:13:05.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-01T00:13:05.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-01T00:13:06.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-01T00:13:06.118 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-01T00:13:06.118 INFO:tasks.ceph:role: osd.6 2024-04-01T00:13:06.118 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:13:06.118 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:13:06.193 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:13:06.194 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:13:06.194 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:13:06.194 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:13:06.680 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-01T00:13:06.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-01T00:13:06.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-01T00:13:06.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-01T00:13:06.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-01T00:13:06.880 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-01T00:13:06.880 INFO:tasks.ceph:role: osd.7 2024-04-01T00:13:06.880 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:13:06.881 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-01T00:13:06.955 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-01T00:13:06.956 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-01T00:13:06.956 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-01T00:13:07.463 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-01T00:13:07.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-01T00:13:07.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-01T00:13:07.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-01T00:13:07.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:07.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.735+0000 7f8bce8d3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-01T00:13:07.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.735+0000 7f8bce8d3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-01T00:13:07.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.735+0000 7f8bce8d3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-01T00:13:07.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.735+0000 7f8bce8d3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-01T00:13:07.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.735+0000 7f8bce8d3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-01T00:13:07.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:07.740+0000 7f8bce8d3640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-01T00:13:10.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-01T00:13:10.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:10.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.707+0000 7fe128810640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-01T00:13:10.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.707+0000 7fe128810640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-01T00:13:10.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.707+0000 7fe128810640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-01T00:13:10.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.707+0000 7fe128810640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-01T00:13:10.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.707+0000 7fe128810640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-01T00:13:10.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:10.708+0000 7fe128810640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-01T00:13:13.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-01T00:13:13.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:13.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.680+0000 7f9423538640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-01T00:13:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.681+0000 7f9423538640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-01T00:13:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.681+0000 7f9423538640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-01T00:13:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.681+0000 7f9423538640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-01T00:13:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.681+0000 7f9423538640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-01T00:13:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:13.681+0000 7f9423538640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-01T00:13:16.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:13:16.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:13:16.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-01T00:13:16.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:16.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.649+0000 7f4a98610640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-01T00:13:16.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.649+0000 7f4a98610640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-01T00:13:16.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.649+0000 7f4a98610640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-01T00:13:16.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.649+0000 7f4a98610640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-01T00:13:16.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.649+0000 7f4a98610640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-01T00:13:16.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:16.650+0000 7f4a98610640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-01T00:13:19.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-01T00:13:19.536 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-01T00:13:19.536 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-01T00:13:19.568 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-01T00:13:19.639 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-01T00:13:19.709 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-01T00:13:19.777 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-01T00:13:19.849 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-01T00:13:19.916 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:19.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-01T00:13:19.985 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:19.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-01T00:13:20.016 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-01T00:13:20.088 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.088 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-01T00:13:20.157 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-01T00:13:20.226 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-01T00:13:20.295 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-01T00:13:20.364 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-01T00:13:20.431 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:20.431 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-01T00:13:20.447 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:20.447 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-01T00:13:20.501 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.501 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-01T00:13:20.517 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:20.517 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-01T00:13:20.572 INFO:tasks.ceph:Adding keys to all mons... 2024-04-01T00:13:20.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-01T00:13:20.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAA/AlmT00PKRAATdhTPJ3xp/s5utlvwwbHnw== 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAA/Almpy5fMBAACQ2Bc0pp2abrcKUna5lLhg== 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAA/Alm5LCsNxAAeCbeyTM3Iye2JyFExe+KEw== 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAE/AlmMBLUKhAAY/UsFVVr2VY84mszKGbiBg== 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAH/AlmJcyLKhAApWlXHrsyypZecQzPJ+PWIw== 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2024-04-01T00:13:20.615 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAK/AlmlGHJKBAAQxxO8RKEod66G8zQUGC3Wg== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAN/AlmX/hdJxAAtY4W+XjlHbJX6UlYGS52nA== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAA/Alm29f6LBAAgGa1IoQeIWAwVGClRmU5/A== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/AlmRpEEARAArkEoLwuviMjZnN98I52POg== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/Almw7deCBAAu7fj48GD+FcAaC8jknSJjQ== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAT/AlmZg3hKxAAmuuwA+m4et4Dgmjb16rnFA== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAW/AlmS+QyKhAA9UEyBinWotqBvdIuBDvAFA== 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2024-04-01T00:13:20.616 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAZ/AlmQKahKBAARv4D/FmeHLlgwfwRogi16A== 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAc/AlmDP/AJhAAXuHiBi9ziGwyJ/3b1R8cKg== 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/AlmQSrFDBAAIkPmgw3eT122JxbfaL7NsQ== 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/Almw3ODEhAAqc3CoCnjNcAllG+5kTidqQ== 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/AlmSVzxFRAAOssJtGlVfNpjHcwnGszLWg== 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2024-04-01T00:13:20.617 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/AlmUn25GxAADK4DgCw2Bxp/M3c/WWUeWg== 2024-04-01T00:13:20.637 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-04-01T00:13:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAA/AlmT00PKRAATdhTPJ3xp/s5utlvwwbHnw== 2024-04-01T00:13:20.637 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-01T00:13:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAA/Almpy5fMBAACQ2Bc0pp2abrcKUna5lLhg== 2024-04-01T00:13:20.637 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAA/Alm5LCsNxAAeCbeyTM3Iye2JyFExe+KEw== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAE/AlmMBLUKhAAY/UsFVVr2VY84mszKGbiBg== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAH/AlmJcyLKhAApWlXHrsyypZecQzPJ+PWIw== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAK/AlmlGHJKBAAQxxO8RKEod66G8zQUGC3Wg== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAN/AlmX/hdJxAAtY4W+XjlHbJX6UlYGS52nA== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAA/Alm29f6LBAAgGa1IoQeIWAwVGClRmU5/A== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/AlmRpEEARAArkEoLwuviMjZnN98I52POg== 2024-04-01T00:13:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/Almw7deCBAAu7fj48GD+FcAaC8jknSJjQ== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAT/AlmZg3hKxAAmuuwA+m4et4Dgmjb16rnFA== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAW/AlmS+QyKhAA9UEyBinWotqBvdIuBDvAFA== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAZ/AlmQKahKBAARv4D/FmeHLlgwfwRogi16A== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAc/AlmDP/AJhAAXuHiBi9ziGwyJ/3b1R8cKg== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/AlmQSrFDBAAIkPmgw3eT122JxbfaL7NsQ== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/Almw3ODEhAAqc3CoCnjNcAllG+5kTidqQ== 2024-04-01T00:13:20.639 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-04-01T00:13:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/AlmSVzxFRAAOssJtGlVfNpjHcwnGszLWg== 2024-04-01T00:13:20.640 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-04-01T00:13:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAB/AlmUn25GxAADK4DgCw2Bxp/M3c/WWUeWg== 2024-04-01T00:13:20.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-01T00:13:20.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-01T00:13:20.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:20.734 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:20.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:20.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:20.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:20.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:20.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:20.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:20.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:20.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-01T00:13:21.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-01T00:13:21.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:21.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:21.174 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:21.176 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-01T00:13:21.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-01T00:13:21.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.573 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.628 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-01T00:13:21.684 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-01T00:13:21.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-01T00:13:21.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-01T00:13:21.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-01T00:13:21.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-01T00:13:21.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-01T00:13:21.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-01T00:13:21.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-01T00:13:22.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-01T00:13:22.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-01T00:13:22.174 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:22.176 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-01T00:13:22.228 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-01T00:13:22.229 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-01T00:13:22.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-01T00:13:22.231 INFO:tasks.ceph.mon.a:Started 2024-04-01T00:13:22.232 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-01T00:13:22.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-01T00:13:22.271 INFO:tasks.ceph.mon.b:Started 2024-04-01T00:13:22.271 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-01T00:13:22.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-01T00:13:22.273 INFO:tasks.ceph.mon.c:Started 2024-04-01T00:13:22.273 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-01T00:13:22.273 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-01T00:13:22.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-01T00:13:22.275 INFO:tasks.ceph.mgr.y:Started 2024-04-01T00:13:22.275 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-01T00:13:22.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-01T00:13:22.277 INFO:tasks.ceph.mgr.x:Started 2024-04-01T00:13:22.277 INFO:tasks.ceph:Setting crush tunables to default 2024-04-01T00:13:22.278 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.377+0000 7fec1fc40640 1 Processor -- start 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.377+0000 7fec1fc40640 1 -- start start 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1fc40640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1fc40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec1815c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1fc40640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec180d0650 con 0x7fec18071c30 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1fc40640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fec18160930 con 0x7fec1815bef0 2024-04-01T00:13:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1fc40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec1815c2d0 con 0x7fec1806b7d0 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44236/0 (socket says 172.21.15.64:44236) 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d1db640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fec18071c30 0x7fec1806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37378/0 (socket says 172.21.15.64:37378) 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 -- 172.21.15.64:0/2256280440 learned_addr learned my addr 172.21.15.64:0/2256280440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 -- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 msgr2=0x7fec181603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 -- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 msgr2=0x7fec181603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 --2- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1d9dc640 1 --2- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec1815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 msgr2=0x7fec1815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 msgr2=0x7fec1815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec1815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-01T00:13:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.378+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec1815c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-01T00:13:22.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.541+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268797443 0 0) 0x7fec180d0650 con 0x7fec18071c30 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec04003520 con 0x7fec18071c30 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1760868306 0 0) 0x7fec04003520 con 0x7fec18071c30 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 msgr2=0x7fec1815c570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec1815c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 msgr2=0x7fec181603f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 --2- 172.21.15.64:0/2256280440 >> v2:172.21.15.64:3300/0 conn(0x7fec1815bef0 0x7fec181603f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec180d0930 con 0x7fec18071c30 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fec14001670 con 0x7fec18071c30 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 >> v1:172.21.15.64:6789/0 conn(0x7fec18071c30 legacy=0x7fec1806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040039e0 0x7fec04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec04004d40 0x7fec040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec18160930 con 0x7fec04004d40 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec1815c2d0 con 0x7fec040039e0 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec0400bb70 con 0x7fec1806b7d0 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec04004d40 0x7fec040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1d9dc640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040039e0 0x7fec04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1d1db640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 msgr2=0x7fec04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040039e0 msgr2=0x7fec04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040039e0 0x7fec04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec04003520 con 0x7fec04004d40 2024-04-01T00:13:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.542+0000 7fec1d1db640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1806b7d0 0x7fec04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.543+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec04004d40 0x7fec040051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec10009610 tx=0x7fec04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc05afcf429ece89 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.543+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10039070 con 0x7fec04004d40 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.543+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1002cc70 con 0x7fec04004d40 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.543+0000 7fec1c9da640 1 -- 172.21.15.64:0/2256280440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10009bd0 con 0x7fec04004d40 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 -- 172.21.15.64:0/2256280440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec04004d40 msgr2=0x7fec040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 --2- 172.21.15.64:0/2256280440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec04004d40 0x7fec040051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec10009610 tx=0x7fec04005830 comp rx=0 tx=0).stop 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1e1dd640 1 -- 172.21.15.64:0/2256280440 reap_dead start 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 -- 172.21.15.64:0/2256280440 shutdown_connections 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 -- 172.21.15.64:0/2256280440 >> 172.21.15.64:0/2256280440 conn(0x7fec180d0c40 msgr2=0x7fec180d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 -- 172.21.15.64:0/2256280440 shutdown_connections 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.544+0000 7fec1fc40640 1 -- 172.21.15.64:0/2256280440 wait complete. 2024-04-01T00:13:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.545+0000 7fec1fc40640 1 Processor -- start 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.545+0000 7fec1fc40640 1 -- start start 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 0x7fec1814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1814df90 0x7fec1814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec180d0340 con 0x7fec181f4fa0 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec180cd2c0 con 0x7fec04004d40 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1fc40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec180ccc20 con 0x7fec1814df90 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1e1dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1e1dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44258/0 (socket says 172.21.15.64:44258) 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1e1dd640 1 -- 172.21.15.64:0/1061869469 learned_addr learned my addr 172.21.15.64:0/1061869469 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1d1db640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1814df90 0x7fec1814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.546+0000 7fec1d9dc640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 0x7fec1814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 -- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1814df90 msgr2=0x7fec1814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1814df90 0x7fec1814d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 -- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 msgr2=0x7fec1814d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 0x7fec1814d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 -- 172.21.15.64:0/1061869469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec180c64e0 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1d9dc640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 0x7fec1814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1e1dd640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec100073f0 tx=0x7fec10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10039040 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec10046070 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec180c67c0 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10009bd0 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec180c6d80 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec1003f470 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.547+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec1004b5a0 con 0x7fec181f4fa0 2024-04-01T00:13:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.548+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec180688b0 con 0x7fec181f4fa0 2024-04-01T00:13:22.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.550+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fec1002ce20 con 0x7fec181f4fa0 2024-04-01T00:13:22.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.640+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fec180c70f0 con 0x7fec181f4fa0 2024-04-01T00:13:22.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.641+0000 7fec0a7fc640 1 -- 172.21.15.64:0/1061869469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fec1003f8a0 con 0x7fec181f4fa0 2024-04-01T00:13:22.642 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 msgr2=0x7fec181f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec100073f0 tx=0x7fec10004ab0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 shutdown_connections 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec181f4fa0 0x7fec181f5430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1814df90 0x7fec1814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 --2- 172.21.15.64:0/1061869469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec04004d40 0x7fec1814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:22.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 >> 172.21.15.64:0/1061869469 conn(0x7fec180d0c40 msgr2=0x7fec180ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:22.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 shutdown_connections 2024-04-01T00:13:22.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:22.642+0000 7fec1fc40640 1 -- 172.21.15.64:0/1061869469 wait complete. 2024-04-01T00:13:22.654 INFO:tasks.ceph:check_enable_crimson: False 2024-04-01T00:13:22.654 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-01T00:13:22.654 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:22.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-01T00:13:22.680 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:22.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-01T00:13:22.747 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:22.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-01T00:13:22.813 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:13:22.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-01T00:13:22.880 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:22.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-01T00:13:22.908 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:22.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-01T00:13:22.975 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:22.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-01T00:13:23.023 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.022+0000 7eff3ea40040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-01T00:13:23.032 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.031+0000 7fac06c41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-01T00:13:23.043 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:13:23.043 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-01T00:13:23.080 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.079+0000 7eff3ea40040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-01T00:13:23.088 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.087+0000 7fac06c41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-01T00:13:23.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 6e752169-6147-48da-8dd8-9a0e36056013 0 2024-04-01T00:13:23.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.245+0000 7eff77d02640 1 Processor -- start 2024-04-01T00:13:23.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.245+0000 7eff77d02640 1 -- start start 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff700d1730 0x7eff7006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff700cfd30 0x7eff700d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7eff700d06e0 con 0x7eff700d1730 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7eff700706b0 con 0x7eff700cfd30 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff77d02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff7006b1f0 con 0x7eff7015c0d0 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff75a9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff700cfd30 0x7eff700d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff75a9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60802/0 (socket says 172.21.15.100:60802) 2024-04-01T00:13:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff75a9e640 1 -- 172.21.15.100:0/233933712 learned_addr learned my addr 172.21.15.100:0/233933712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7529d640 1 --2- 172.21.15.100:0/233933712 >> v2:172.21.15.64:3300/0 conn(0x7eff700d1730 0x7eff7006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 -- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 msgr2=0x7eff700726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 -- 172.21.15.100:0/233933712 >> v2:172.21.15.64:3300/0 conn(0x7eff700d1730 msgr2=0x7eff7006b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 --2- 172.21.15.100:0/233933712 >> v2:172.21.15.64:3300/0 conn(0x7eff700d1730 0x7eff7006b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff7629f640 1 -- 172.21.15.100:0/233933712 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff700700f0 con 0x7eff700cfd30 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.246+0000 7eff75a9e640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.247+0000 7eff7629f640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700cfd30 0x7eff700d01a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff70067e90 tx=0x7eff6802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92b8776ace98678 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.247+0000 7eff74a9c640 1 -- 172.21.15.100:0/233933712 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68039070 con 0x7eff700cfd30 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.247+0000 7eff74a9c640 1 -- 172.21.15.100:0/233933712 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6802cb40 con 0x7eff700cfd30 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.247+0000 7eff74a9c640 1 -- 172.21.15.100:0/233933712 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68034440 con 0x7eff700cfd30 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 -- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700cfd30 msgr2=0x7eff700d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700cfd30 0x7eff700d01a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff70067e90 tx=0x7eff6802bd90 comp rx=0 tx=0).stop 2024-04-01T00:13:23.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 -- 172.21.15.100:0/233933712 shutdown_connections 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700cfd30 0x7eff700d01a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 --2- 172.21.15.100:0/233933712 >> v2:172.21.15.64:3300/0 conn(0x7eff700d1730 0x7eff7006b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 --2- 172.21.15.100:0/233933712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff7015c0d0 0x7eff700726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 -- 172.21.15.100:0/233933712 >> 172.21.15.100:0/233933712 conn(0x7eff700d0c10 msgr2=0x7eff700d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 -- 172.21.15.100:0/233933712 shutdown_connections 2024-04-01T00:13:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.248+0000 7eff77d02640 1 -- 172.21.15.100:0/233933712 wait complete. 2024-04-01T00:13:23.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.249+0000 7eff77d02640 1 Processor -- start 2024-04-01T00:13:23.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.249+0000 7eff77d02640 1 -- start start 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.249+0000 7eff77d02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 0x7eff7009aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.249+0000 7eff77d02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff7006bef0 con 0x7eff700cfd30 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff7006b490 con 0x7eff7015c0d0 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff7015c4b0 con 0x7eff700d1730 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7529d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32940/0 (socket says 172.21.15.100:32940) 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7529d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60804/0 (socket says 172.21.15.100:60804) 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 -- 172.21.15.100:0/90292038 learned_addr learned my addr 172.21.15.100:0/90292038 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff75a9e640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 0x7eff7009aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 -- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 msgr2=0x7eff701487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 -- 172.21.15.100:0/90292038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 msgr2=0x7eff7009aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 0x7eff7009aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 -- 172.21.15.100:0/90292038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701f59f0 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7529d640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff75a9e640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 0x7eff7009aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff7629f640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff68009e80 tx=0x7eff68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68039050 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68045070 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701f5cd0 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68009450 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701f6290 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff6803e850 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.250+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7eff6804a9d0 con 0x7eff7015c0d0 2024-04-01T00:13:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.251+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff700d12b0 con 0x7eff7015c0d0 2024-04-01T00:13:23.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.253+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7eff700d12b0 con 0x7eff7015c0d0 2024-04-01T00:13:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.324+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e752169-6147-48da-8dd8-9a0e36056013", "id": 0} v 0) -- 0x7eff70148d30 con 0x7eff7015c0d0 2024-04-01T00:13:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.330+0000 7eff667fc640 1 -- 172.21.15.100:0/90292038 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e752169-6147-48da-8dd8-9a0e36056013", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7eff6804a460 con 0x7eff7015c0d0 2024-04-01T00:13:23.331 INFO:teuthology.orchestra.run.smithi100.stdout:0 2024-04-01T00:13:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 msgr2=0x7eff701454b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff68009e80 tx=0x7eff68004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 shutdown_connections 2024-04-01T00:13:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7015c0d0 0x7eff701454b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff700d1730 0x7eff701487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 --2- 172.21.15.100:0/90292038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff700cfd30 0x7eff7009aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 >> 172.21.15.100:0/90292038 conn(0x7eff700d0c10 msgr2=0x7eff7006f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 shutdown_connections 2024-04-01T00:13:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.331+0000 7eff77d02640 1 -- 172.21.15.100:0/90292038 wait complete. 2024-04-01T00:13:23.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new ade989fe-45b3-4539-a94f-d87f9387130b 1 2024-04-01T00:13:23.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.442+0000 7fbe09301640 1 Processor -- start 2024-04-01T00:13:23.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.442+0000 7fbe09301640 1 -- start start 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 0x7fbe04160520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 0x7fbe040d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fbe040b53f0 con 0x7fbe040ceaf0 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbe040d0ce0 con 0x7fbe0415bc70 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe09301640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbe04160a60 con 0x7fbe040d0370 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe02ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 0x7fbe040d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe02ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32946/0 (socket says 172.21.15.100:32946) 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe02ffd640 1 -- 172.21.15.100:0/2370538389 learned_addr learned my addr 172.21.15.100:0/2370538389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe027fc640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 0x7fbe04160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 -- 172.21.15.100:0/2370538389 >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 msgr2=0x7fbe0415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 -- 172.21.15.100:0/2370538389 >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 msgr2=0x7fbe04160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 0x7fbe04160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe037fe640 1 -- 172.21.15.100:0/2370538389 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe040cf120 con 0x7fbe040d0370 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe02ffd640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.443+0000 7fbe027fc640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 0x7fbe04160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.444+0000 7fbe037fe640 1 --2- 172.21.15.100:0/2370538389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 0x7fbe040d07a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe04072810 tx=0x7fbdfc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5440799358dafc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.444+0000 7fbe01ffb640 1 -- 172.21.15.100:0/2370538389 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc039070 con 0x7fbe040d0370 2024-04-01T00:13:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.444+0000 7fbe01ffb640 1 -- 172.21.15.100:0/2370538389 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc02cbc0 con 0x7fbe040d0370 2024-04-01T00:13:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.444+0000 7fbe01ffb640 1 -- 172.21.15.100:0/2370538389 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc034440 con 0x7fbe040d0370 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 -- 172.21.15.100:0/2370538389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 msgr2=0x7fbe040d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 --2- 172.21.15.100:0/2370538389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 0x7fbe040d07a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe04072810 tx=0x7fbdfc02be10 comp rx=0 tx=0).stop 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 -- 172.21.15.100:0/2370538389 shutdown_connections 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 --2- 172.21.15.100:0/2370538389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe040d0370 0x7fbe040d07a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.64:3300/0 conn(0x7fbe040ceaf0 0x7fbe04160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 --2- 172.21.15.100:0/2370538389 >> v2:172.21.15.100:3300/0 conn(0x7fbe0415bc70 0x7fbe0415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 -- 172.21.15.100:0/2370538389 >> 172.21.15.100:0/2370538389 conn(0x7fbe0402ceb0 msgr2=0x7fbe0406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 -- 172.21.15.100:0/2370538389 shutdown_connections 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.445+0000 7fbe09301640 1 -- 172.21.15.100:0/2370538389 wait complete. 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.446+0000 7fbe09301640 1 Processor -- start 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.446+0000 7fbe09301640 1 -- start start 2024-04-01T00:13:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe040d0370 0x7fbe04146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 0x7fbe0406dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040cf9c0 con 0x7fbe040d0370 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe040cf6e0 con 0x7fbe040ceaf0 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe09301640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbe040cfca0 con 0x7fbe0415bc70 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 0x7fbe0406dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32950/0 (socket says 172.21.15.100:32950) 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe02ffd640 1 -- 172.21.15.100:0/3998529039 learned_addr learned my addr 172.21.15.100:0/3998529039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe027fc640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe040d0370 0x7fbe04146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 -- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 msgr2=0x7fbe040c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 -- 172.21.15.100:0/3998529039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe040d0370 msgr2=0x7fbe04146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe040d0370 0x7fbe04146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 -- 172.21.15.100:0/3998529039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0406e1e0 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe02ffd640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.447+0000 7fbe037fe640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 0x7fbe0406dca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc033000 tx=0x7fbdfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc039050 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc045070 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe04153950 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc009450 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe04153f10 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbdfc03e850 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.448+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbdfc04add0 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.449+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe040688b0 con 0x7fbe0415bc70 2024-04-01T00:13:23.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.451+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbdfc04a460 con 0x7fbe0415bc70 2024-04-01T00:13:23.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.524+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ade989fe-45b3-4539-a94f-d87f9387130b", "id": 1} v 0) -- 0x7fbe0402c290 con 0x7fbe0415bc70 2024-04-01T00:13:23.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.531+0000 7fbdeb7fe640 1 -- 172.21.15.100:0/3998529039 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ade989fe-45b3-4539-a94f-d87f9387130b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbdfc03ec30 con 0x7fbe0415bc70 2024-04-01T00:13:23.532 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-01T00:13:23.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 msgr2=0x7fbe0406dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 0x7fbe0406dca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc033000 tx=0x7fbdfc004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 shutdown_connections 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe0415bc70 0x7fbe0406dca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe040d0370 0x7fbe04146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 --2- 172.21.15.100:0/3998529039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe040ceaf0 0x7fbe040c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 >> 172.21.15.100:0/3998529039 conn(0x7fbe0402ceb0 msgr2=0x7fbe040b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 shutdown_connections 2024-04-01T00:13:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.532+0000 7fbe09301640 1 -- 172.21.15.100:0/3998529039 wait complete. 2024-04-01T00:13:23.544 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new ea0a8972-b335-43d1-b796-294fe768a068 2 2024-04-01T00:13:23.632 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.631+0000 7fac06c41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-01T00:13:23.637 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.636+0000 7eff3ea40040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-01T00:13:23.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.644+0000 7fe26c9c2640 1 Processor -- start 2024-04-01T00:13:23.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.644+0000 7fe26c9c2640 1 -- start start 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26c9c2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe26815f4a0 0x7fe2680cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26c9c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 0x7fe2680cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26c9c2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2680cdaf0 con 0x7fe2680cf330 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26c9c2640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe2680d2aa0 con 0x7fe26815f4a0 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26c9c2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe2680ce0c0 con 0x7fe26815f880 2024-04-01T00:13:23.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe266d9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe2680cf330 0x7fe2680b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46048/0 (socket says 172.21.15.100:46048) 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe265d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 0x7fe2680cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe266d9d640 1 -- 172.21.15.100:0/3536173544 learned_addr learned my addr 172.21.15.100:0/3536173544 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26659c640 1 --2- 172.21.15.100:0/3536173544 >> v2:172.21.15.64:3300/0 conn(0x7fe26815f4a0 0x7fe2680cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe26559a640 1 -- 172.21.15.100:0/3536173544 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183829410 0 0) 0x7fe2680cdaf0 con 0x7fe2680cf330 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe265d9b640 1 -- 172.21.15.100:0/3536173544 >> v2:172.21.15.64:3300/0 conn(0x7fe26815f4a0 msgr2=0x7fe2680cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe265d9b640 1 --2- 172.21.15.100:0/3536173544 >> v2:172.21.15.64:3300/0 conn(0x7fe26815f4a0 0x7fe2680cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe265d9b640 1 -- 172.21.15.100:0/3536173544 >> v1:172.21.15.100:6789/0 conn(0x7fe2680cf330 legacy=0x7fe2680b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.645+0000 7fe265d9b640 1 -- 172.21.15.100:0/3536173544 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2680ce310 con 0x7fe26815f880 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.646+0000 7fe265d9b640 1 --2- 172.21.15.100:0/3536173544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 0x7fe2680cedf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53175272f3f00a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.646+0000 7fe26559a640 1 -- 172.21.15.100:0/3536173544 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0315a0 con 0x7fe26815f880 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.646+0000 7fe26559a640 1 -- 172.21.15.100:0/3536173544 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c037070 con 0x7fe26815f880 2024-04-01T00:13:23.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.646+0000 7fe26559a640 1 -- 172.21.15.100:0/3536173544 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c031960 con 0x7fe26815f880 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 -- 172.21.15.100:0/3536173544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 msgr2=0x7fe2680cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/3536173544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 0x7fe2680cedf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02b840 comp rx=0 tx=0).stop 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 -- 172.21.15.100:0/3536173544 shutdown_connections 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/3536173544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe26815f880 0x7fe2680cedf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/3536173544 >> v2:172.21.15.64:3300/0 conn(0x7fe26815f4a0 0x7fe2680cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 -- 172.21.15.100:0/3536173544 >> 172.21.15.100:0/3536173544 conn(0x7fe2680d1d10 msgr2=0x7fe2680d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 -- 172.21.15.100:0/3536173544 shutdown_connections 2024-04-01T00:13:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.647+0000 7fe26c9c2640 1 -- 172.21.15.100:0/3536173544 wait complete. 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.648+0000 7fe26c9c2640 1 Processor -- start 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.648+0000 7fe26c9c2640 1 -- start start 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 0x7fe268153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 0x7fe268157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26815f880 0x7fe2681553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2680b1bb0 con 0x7fe26815f880 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe2680ce960 con 0x7fe26815f4a0 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26c9c2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe2680b1d30 con 0x7fe2680cf330 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 0x7fe268153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 0x7fe268157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 0x7fe268153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60848/0 (socket says 172.21.15.100:60848) 2024-04-01T00:13:23.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe26659c640 1 -- 172.21.15.100:0/2065852120 learned_addr learned my addr 172.21.15.100:0/2065852120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe266d9d640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26815f880 0x7fe2681553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 -- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 msgr2=0x7fe268153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 0x7fe268153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 -- 172.21.15.100:0/2065852120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26815f880 msgr2=0x7fe2681553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26815f880 0x7fe2681553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.649+0000 7fe265d9b640 1 -- 172.21.15.100:0/2065852120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2680c8de0 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe265d9b640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 0x7fe268157f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe25c033000 tx=0x7fe25c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c032630 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c0327f0 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c032be0 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2680c90c0 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2680c9680 con 0x7fe26815f4a0 2024-04-01T00:13:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe25c04e7c0 con 0x7fe26815f4a0 2024-04-01T00:13:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.650+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fe25c02f030 con 0x7fe26815f4a0 2024-04-01T00:13:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.651+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2680688b0 con 0x7fe26815f4a0 2024-04-01T00:13:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.653+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe25c040070 con 0x7fe26815f4a0 2024-04-01T00:13:23.688 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.688+0000 7fac06c41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-01T00:13:23.695 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.694+0000 7eff3ea40040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-01T00:13:23.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.725+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea0a8972-b335-43d1-b796-294fe768a068", "id": 2} v 0) -- 0x7fe2680b7c90 con 0x7fe26815f4a0 2024-04-01T00:13:23.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.731+0000 7fe256ffd640 1 -- 172.21.15.100:0/2065852120 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea0a8972-b335-43d1-b796-294fe768a068", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fe25c032d80 con 0x7fe26815f4a0 2024-04-01T00:13:23.733 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-04-01T00:13:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 msgr2=0x7fe268157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 0x7fe268157f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe25c033000 tx=0x7fe25c0046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 shutdown_connections 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26815f880 0x7fe2681553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe26815f4a0 0x7fe268157f80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 --2- 172.21.15.100:0/2065852120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2680cf330 0x7fe268153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 >> 172.21.15.100:0/2065852120 conn(0x7fe2680d1d10 msgr2=0x7fe2680b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 shutdown_connections 2024-04-01T00:13:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.732+0000 7fe26c9c2640 1 -- 172.21.15.100:0/2065852120 wait complete. 2024-04-01T00:13:23.735 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.735+0000 7fac06c41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-01T00:13:23.742 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.741+0000 7eff3ea40040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-01T00:13:23.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 3f21b4bf-b9dd-4772-a5de-ea0819be0b07 3 2024-04-01T00:13:23.791 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.790+0000 7fac06c41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-01T00:13:23.800 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.799+0000 7eff3ea40040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-01T00:13:23.835 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.834+0000 7fac06c41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-01T00:13:23.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.843+0000 7f3d81ec0640 1 Processor -- start 2024-04-01T00:13:23.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.843+0000 7f3d81ec0640 1 -- start start 2024-04-01T00:13:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d81ec0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d81ec0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d7c0cebf0 0x7f3d7c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d81ec0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7c0cd440 con 0x7f3d7c0ccca0 2024-04-01T00:13:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d81ec0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7c0cf590 con 0x7f3d7c0cebf0 2024-04-01T00:13:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d81ec0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7c0cf050 con 0x7f3d7c15c060 2024-04-01T00:13:23.845 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.844+0000 7eff3ea40040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-01T00:13:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60862/0 (socket says 172.21.15.100:60862) 2024-04-01T00:13:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 -- 172.21.15.100:0/3081956373 learned_addr learned my addr 172.21.15.100:0/3081956373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d72ffd640 1 --2- 172.21.15.100:0/3081956373 >> v2:172.21.15.64:3300/0 conn(0x7f3d7c0cebf0 0x7f3d7c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 -- 172.21.15.100:0/3081956373 >> v2:172.21.15.64:3300/0 conn(0x7f3d7c0cebf0 msgr2=0x7f3d7c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 --2- 172.21.15.100:0/3081956373 >> v2:172.21.15.64:3300/0 conn(0x7f3d7c0cebf0 0x7f3d7c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 -- 172.21.15.100:0/3081956373 >> v1:172.21.15.64:6789/0 conn(0x7f3d7c0ccca0 legacy=0x7f3d7c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.844+0000 7f3d7b7fe640 1 -- 172.21.15.100:0/3081956373 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c0cd720 con 0x7f3d7c15c060 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.845+0000 7f3d7b7fe640 1 --2- 172.21.15.100:0/3081956373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009870 tx=0x7f3d6c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daadcad3250a4c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.845+0000 7f3d7affd640 1 -- 172.21.15.100:0/3081956373 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c02f840 con 0x7f3d7c15c060 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.845+0000 7f3d7affd640 1 -- 172.21.15.100:0/3081956373 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6c02f9e0 con 0x7f3d7c15c060 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.845+0000 7f3d7affd640 1 -- 172.21.15.100:0/3081956373 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c02fdb0 con 0x7f3d7c15c060 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 -- 172.21.15.100:0/3081956373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 msgr2=0x7f3d7c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/3081956373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009870 tx=0x7f3d6c02ba20 comp rx=0 tx=0).stop 2024-04-01T00:13:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 -- 172.21.15.100:0/3081956373 shutdown_connections 2024-04-01T00:13:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/3081956373 >> v2:172.21.15.64:3300/0 conn(0x7f3d7c0cebf0 0x7f3d7c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/3081956373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c15c060 0x7f3d7c0ce6b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 -- 172.21.15.100:0/3081956373 >> 172.21.15.100:0/3081956373 conn(0x7f3d7c09a980 msgr2=0x7f3d7c09ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 -- 172.21.15.100:0/3081956373 shutdown_connections 2024-04-01T00:13:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.846+0000 7f3d81ec0640 1 -- 172.21.15.100:0/3081956373 wait complete. 2024-04-01T00:13:23.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.847+0000 7f3d81ec0640 1 Processor -- start 2024-04-01T00:13:23.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.847+0000 7f3d81ec0640 1 -- start start 2024-04-01T00:13:23.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d7c0ccca0 0x7f3d7c15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c0cebf0 0x7f3d7c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c0cdfc0 con 0x7f3d7c0ccca0 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c0cdce0 con 0x7f3d7c15c060 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d81ec0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7c0cd120 con 0x7f3d7c0cebf0 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d72ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c0cebf0 0x7f3d7c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32966/0 (socket says 172.21.15.100:32966) 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 -- 172.21.15.100:0/4176608820 learned_addr learned my addr 172.21.15.100:0/4176608820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7b7fe640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d7c0ccca0 0x7f3d7c15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 -- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c0cebf0 msgr2=0x7f3d7c151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c0cebf0 0x7f3d7c151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 -- 172.21.15.100:0/4176608820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d7c0ccca0 msgr2=0x7f3d7c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d7c0ccca0 0x7f3d7c15d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 -- 172.21.15.100:0/4176608820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c15f410 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.848+0000 7f3d7bfff640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c730 tx=0x7f3d7400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74003b70 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d74003d10 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c0b5c30 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400aa90 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c15f6f0 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d7400ac30 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.849+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3d74012a40 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.850+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7c0688b0 con 0x7f3d7c15c060 2024-04-01T00:13:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.852+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3d74012ce0 con 0x7f3d7c15c060 2024-04-01T00:13:23.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.924+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3f21b4bf-b9dd-4772-a5de-ea0819be0b07", "id": 3} v 0) -- 0x7f3d7c09b5f0 con 0x7f3d7c15c060 2024-04-01T00:13:23.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.930+0000 7f3d78ff9640 1 -- 172.21.15.100:0/4176608820 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f21b4bf-b9dd-4772-a5de-ea0819be0b07", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3d740125b0 con 0x7f3d7c15c060 2024-04-01T00:13:23.931 INFO:teuthology.orchestra.run.smithi100.stdout:3 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 msgr2=0x7f3d7c15eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c730 tx=0x7f3d7400cbf0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 shutdown_connections 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c15c060 0x7f3d7c15eac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7c0cebf0 0x7f3d7c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 --2- 172.21.15.100:0/4176608820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d7c0ccca0 0x7f3d7c15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 >> 172.21.15.100:0/4176608820 conn(0x7f3d7c09a980 msgr2=0x7f3d7c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 shutdown_connections 2024-04-01T00:13:23.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:23.931+0000 7f3d81ec0640 1 -- 172.21.15.100:0/4176608820 wait complete. 2024-04-01T00:13:23.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 0f18736e-7c9c-424e-b829-ab17c5a8b64d 4 2024-04-01T00:13:23.955 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:23.955+0000 7fac06c41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-01T00:13:23.968 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:23.967+0000 7eff3ea40040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-01T00:13:24.028 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.027+0000 7fac06c41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-01T00:13:24.041 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.041+0000 7eff3ea40040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-01T00:13:24.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 Processor -- start 2024-04-01T00:13:24.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 -- start start 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff4c15e340 0x7eff4c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 0x7eff4c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4c06f4a0 con 0x7eff4c0cec70 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7eff4c0cf0d0 con 0x7eff4c15e340 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.042+0000 7eff5174b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c06df50 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff4c15e340 0x7eff4c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 0x7eff4c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff4c15e340 0x7eff4c06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32978/0 (socket says 172.21.15.100:32978) 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4b7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7eff4c0cec70 0x7eff4c15ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46054/0 (socket says 172.21.15.100:46054) 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 -- 172.21.15.100:0/593471280 learned_addr learned my addr 172.21.15.100:0/593471280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 -- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 msgr2=0x7eff4c0ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 0x7eff4c0ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 -- 172.21.15.100:0/593471280 >> v1:172.21.15.100:6789/0 conn(0x7eff4c0cec70 legacy=0x7eff4c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4affd640 1 -- 172.21.15.100:0/593471280 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c06e1a0 con 0x7eff4c15e340 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff49ffb640 1 -- 172.21.15.100:0/593471280 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731467143 0 0) 0x7eff4c06f4a0 con 0x7eff4c0cec70 2024-04-01T00:13:24.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.043+0000 7eff4a7fc640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 0x7eff4c0ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff4affd640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c15e340 0x7eff4c06ef60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009870 tx=0x7eff3c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f3fedb29f589b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff49ffb640 1 -- 172.21.15.100:0/593471280 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035020 con 0x7eff4c15e340 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff49ffb640 1 -- 172.21.15.100:0/593471280 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c031640 con 0x7eff4c15e340 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff49ffb640 1 -- 172.21.15.100:0/593471280 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c02d040 con 0x7eff4c15e340 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 -- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c15e340 msgr2=0x7eff4c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c15e340 0x7eff4c06ef60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009870 tx=0x7eff3c02b8f0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 -- 172.21.15.100:0/593471280 shutdown_connections 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0ce2d0 0x7eff4c0ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 --2- 172.21.15.100:0/593471280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c15e340 0x7eff4c06ef60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 -- 172.21.15.100:0/593471280 >> 172.21.15.100:0/593471280 conn(0x7eff4c072890 msgr2=0x7eff4c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.044+0000 7eff5174b640 1 -- 172.21.15.100:0/593471280 shutdown_connections 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.045+0000 7eff5174b640 1 -- 172.21.15.100:0/593471280 wait complete. 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 Processor -- start 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 -- start start 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 0x7eff4c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff4c15e340 0x7eff4c0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c06e760 con 0x7eff4c15e340 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c0ad220 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.046+0000 7eff5174b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c15e720 con 0x7eff4c0cec70 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 0x7eff4c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32980/0 (socket says 172.21.15.100:32980) 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 0x7eff4c0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60884/0 (socket says 172.21.15.100:60884) 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 -- 172.21.15.100:0/1965011096 learned_addr learned my addr 172.21.15.100:0/1965011096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4b7fe640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff4c15e340 0x7eff4c0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 -- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 msgr2=0x7eff4c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 0x7eff4c0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 -- 172.21.15.100:0/1965011096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff4c15e340 msgr2=0x7eff4c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff4c15e340 0x7eff4c0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 -- 172.21.15.100:0/1965011096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c0b4830 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff4affd640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c00 tx=0x7eff3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c040070 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c0399f0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c0b4ad0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035040 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c0b5090 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff3c04b060 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.047+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7eff3c03a930 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.048+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c0688b0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.050+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7eff3c03abd0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.076 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.075+0000 7fac06c41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-01T00:13:24.091 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.090+0000 7eff3ea40040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-01T00:13:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.123+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0f18736e-7c9c-424e-b829-ab17c5a8b64d", "id": 4} v 0) -- 0x7eff4c0720b0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.128+0000 7eff337fe640 1 -- 172.21.15.100:0/1965011096 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f18736e-7c9c-424e-b829-ab17c5a8b64d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7eff3c03adb0 con 0x7eff4c0ce2d0 2024-04-01T00:13:24.129 INFO:teuthology.orchestra.run.smithi100.stdout:4 2024-04-01T00:13:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 msgr2=0x7eff4c1e4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c00 tx=0x7eff3c0046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 shutdown_connections 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff4c15e340 0x7eff4c0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff4c0cec70 0x7eff4c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 --2- 172.21.15.100:0/1965011096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4c0ce2d0 0x7eff4c1e4ff0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 >> 172.21.15.100:0/1965011096 conn(0x7eff4c072890 msgr2=0x7eff4c0ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 shutdown_connections 2024-04-01T00:13:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.129+0000 7eff5174b640 1 -- 172.21.15.100:0/1965011096 wait complete. 2024-04-01T00:13:24.131 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.130+0000 7fac06c41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-01T00:13:24.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 9c19a862-8d4e-4a8d-9efb-c44c4f905fe3 5 2024-04-01T00:13:24.146 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.146+0000 7eff3ea40040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-01T00:13:24.179 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.179+0000 7fac06c41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-01T00:13:24.197 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.196+0000 7eff3ea40040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-01T00:13:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.245+0000 7f2027ea4640 1 Processor -- start 2024-04-01T00:13:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 -- start start 2024-04-01T00:13:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 0x7f20200d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202015eda0 con 0x7f20200d0590 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f20200d09f0 con 0x7f20200724b0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2027ea4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f202015f9b0 con 0x7f202015aef0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2026441640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2026441640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32984/0 (socket says 172.21.15.100:32984) 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2025c40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 0x7f20200d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.246+0000 7f2026441640 1 -- 172.21.15.100:0/2677454946 learned_addr learned my addr 172.21.15.100:0/2677454946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.247+0000 7f2025c40640 1 -- 172.21.15.100:0/2677454946 >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 msgr2=0x7f20200aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.247+0000 7f2025c40640 1 --2- 172.21.15.100:0/2677454946 >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.247+0000 7f2025c40640 1 -- 172.21.15.100:0/2677454946 >> v1:172.21.15.64:6789/0 conn(0x7f20200d0590 legacy=0x7f2020071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.247+0000 7f2025c40640 1 -- 172.21.15.100:0/2677454946 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202015f080 con 0x7f202015aef0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.247+0000 7f2026441640 1 --2- 172.21.15.100:0/2677454946 >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2025c40640 1 --2- 172.21.15.100:0/2677454946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 0x7f20200d0050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f201c0097a0 tx=0x7f201c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f96442b86705a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2024c3e640 1 -- 172.21.15.100:0/2677454946 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c035020 con 0x7f202015aef0 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2024c3e640 1 -- 172.21.15.100:0/2677454946 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201c031810 con 0x7f202015aef0 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2024c3e640 1 -- 172.21.15.100:0/2677454946 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c031be0 con 0x7f202015aef0 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 -- 172.21.15.100:0/2677454946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 msgr2=0x7f20200d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 --2- 172.21.15.100:0/2677454946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 0x7f20200d0050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f201c0097a0 tx=0x7f201c02b8b0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 -- 172.21.15.100:0/2677454946 shutdown_connections 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 --2- 172.21.15.100:0/2677454946 >> v2:172.21.15.100:3300/0 conn(0x7f20200724b0 0x7f20200aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 --2- 172.21.15.100:0/2677454946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f202015aef0 0x7f20200d0050 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 -- 172.21.15.100:0/2677454946 >> 172.21.15.100:0/2677454946 conn(0x7f20200d0c70 msgr2=0x7f20200d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 -- 172.21.15.100:0/2677454946 shutdown_connections 2024-04-01T00:13:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.248+0000 7f2027ea4640 1 -- 172.21.15.100:0/2677454946 wait complete. 2024-04-01T00:13:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 Processor -- start 2024-04-01T00:13:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 -- start start 2024-04-01T00:13:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20200724b0 0x7f20201f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 0x7f20200c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f202015b2d0 con 0x7f20200724b0 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f202015f640 con 0x7f202015aef0 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2027ea4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f202015b450 con 0x7f20200d0590 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f202543f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2026441640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 0x7f20200c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2026441640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 0x7f20200c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33000/0 (socket says 172.21.15.100:33000) 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f202543f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60896/0 (socket says 172.21.15.100:60896) 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2026441640 1 -- 172.21.15.100:0/1339610284 learned_addr learned my addr 172.21.15.100:0/1339610284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.250+0000 7f2025c40640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20200724b0 0x7f20201f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 -- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 msgr2=0x7f20200c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 0x7f20200c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 -- 172.21.15.100:0/1339610284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20200724b0 msgr2=0x7f20201f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20200724b0 0x7f20201f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 -- 172.21.15.100:0/1339610284 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200b3d10 con 0x7f20200d0590 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f202543f640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f201000c340 tx=0x7f201000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20100052d0 con 0x7f20200d0590 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010005490 con 0x7f20200d0590 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20200b60d0 con 0x7f20200d0590 2024-04-01T00:13:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010007260 con 0x7f20200d0590 2024-04-01T00:13:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20200b6690 con 0x7f20200d0590 2024-04-01T00:13:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2010007400 con 0x7f20200d0590 2024-04-01T00:13:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.251+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2010011590 con 0x7f20200d0590 2024-04-01T00:13:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.252+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200688b0 con 0x7f20200d0590 2024-04-01T00:13:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.254+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2010003a50 con 0x7f20200d0590 2024-04-01T00:13:24.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.327+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c19a862-8d4e-4a8d-9efb-c44c4f905fe3", "id": 5} v 0) -- 0x7f20200676b0 con 0x7f20200d0590 2024-04-01T00:13:24.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.332+0000 7f20167fc640 1 -- 172.21.15.100:0/1339610284 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c19a862-8d4e-4a8d-9efb-c44c4f905fe3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2010003c30 con 0x7f20200d0590 2024-04-01T00:13:24.333 INFO:teuthology.orchestra.run.smithi100.stdout:5 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 msgr2=0x7f20200b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f201000c340 tx=0x7f201000c800 comp rx=0 tx=0).stop 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 shutdown_connections 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015aef0 0x7f20200c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20200d0590 0x7f20200b37d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 --2- 172.21.15.100:0/1339610284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20200724b0 0x7f20201f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 >> 172.21.15.100:0/1339610284 conn(0x7f20200d0c70 msgr2=0x7f20200afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 shutdown_connections 2024-04-01T00:13:24.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.333+0000 7f2027ea4640 1 -- 172.21.15.100:0/1339610284 wait complete. 2024-04-01T00:13:24.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new d7a44ad8-6b22-4573-a0da-8fb58d1c82a8 6 2024-04-01T00:13:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.443+0000 7f9a73769640 1 Processor -- start 2024-04-01T00:13:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.443+0000 7f9a73769640 1 -- start start 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a73769640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a73769640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a6c06b600 0x7f9a6c072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a73769640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6c15ca20 con 0x7f9a6c0ae5a0 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a73769640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f9a6c15d5a0 con 0x7f9a6c06b600 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a73769640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c15cd00 con 0x7f9a6c0afea0 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60912/0 (socket says 172.21.15.100:60912) 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 -- 172.21.15.100:0/1202582443 learned_addr learned my addr 172.21.15.100:0/1202582443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a71d06640 1 --2- 172.21.15.100:0/1202582443 >> v2:172.21.15.64:3300/0 conn(0x7f9a6c06b600 0x7f9a6c072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 -- 172.21.15.100:0/1202582443 >> v2:172.21.15.64:3300/0 conn(0x7f9a6c06b600 msgr2=0x7f9a6c072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 --2- 172.21.15.100:0/1202582443 >> v2:172.21.15.64:3300/0 conn(0x7f9a6c06b600 0x7f9a6c072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 -- 172.21.15.100:0/1202582443 >> v1:172.21.15.64:6789/0 conn(0x7f9a6c0ae5a0 legacy=0x7f9a6c06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.444+0000 7f9a70d04640 1 -- 172.21.15.100:0/1202582443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c15cfe0 con 0x7f9a6c0afea0 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.445+0000 7f9a70d04640 1 --2- 172.21.15.100:0/1202582443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009870 tx=0x7f9a6802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538ea2bfa4846629 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.445+0000 7f9a5ffff640 1 -- 172.21.15.100:0/1202582443 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68034070 con 0x7f9a6c0afea0 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.445+0000 7f9a5ffff640 1 -- 172.21.15.100:0/1202582443 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a68030840 con 0x7f9a6c0afea0 2024-04-01T00:13:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.445+0000 7f9a5ffff640 1 -- 172.21.15.100:0/1202582443 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68030c10 con 0x7f9a6c0afea0 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 -- 172.21.15.100:0/1202582443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 msgr2=0x7f9a6c06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 --2- 172.21.15.100:0/1202582443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009870 tx=0x7f9a6802b980 comp rx=0 tx=0).stop 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 -- 172.21.15.100:0/1202582443 shutdown_connections 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 --2- 172.21.15.100:0/1202582443 >> v2:172.21.15.64:3300/0 conn(0x7f9a6c06b600 0x7f9a6c072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 --2- 172.21.15.100:0/1202582443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c0afea0 0x7f9a6c06b0c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 -- 172.21.15.100:0/1202582443 >> 172.21.15.100:0/1202582443 conn(0x7f9a6c0d0c70 msgr2=0x7f9a6c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 -- 172.21.15.100:0/1202582443 shutdown_connections 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.446+0000 7f9a73769640 1 -- 172.21.15.100:0/1202582443 wait complete. 2024-04-01T00:13:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.447+0000 7f9a73769640 1 Processor -- start 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.447+0000 7f9a73769640 1 -- start start 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 0x7f9a6c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0afea0 0x7f9a6c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c0b0300 con 0x7f9a6c0afea0 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c0ae980 con 0x7f9a6c0ae5a0 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c15c680 con 0x7f9a6c06b600 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a70d04640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 0x7f9a6c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60926/0 (socket says 172.21.15.100:60926) 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 -- 172.21.15.100:0/4289484004 learned_addr learned my addr 172.21.15.100:0/4289484004 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71d06640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0afea0 0x7f9a6c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 -- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 msgr2=0x7f9a6c0c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 0x7f9a6c0c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 -- 172.21.15.100:0/4289484004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0afea0 msgr2=0x7f9a6c154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0afea0 0x7f9a6c154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 -- 172.21.15.100:0/4289484004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c1548d0 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a70d04640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 0x7f9a6c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a71505640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c400 tx=0x7f9a6000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600052d0 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a60005490 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6c153a40 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.448+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60007260 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.449+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6c153480 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.449+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a60007400 con 0x7f9a6c06b600 2024-04-01T00:13:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.449+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9a60011590 con 0x7f9a6c06b600 2024-04-01T00:13:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.450+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6c0b53c0 con 0x7f9a6c06b600 2024-04-01T00:13:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.452+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9a6c1548d0 con 0x7f9a6c06b600 2024-04-01T00:13:24.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.523+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d7a44ad8-6b22-4573-a0da-8fb58d1c82a8", "id": 6} v 0) -- 0x7f9a6c0688b0 con 0x7f9a6c06b600 2024-04-01T00:13:24.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.528+0000 7f9a5dffb640 1 -- 172.21.15.100:0/4289484004 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7a44ad8-6b22-4573-a0da-8fb58d1c82a8", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9a60003a00 con 0x7f9a6c06b600 2024-04-01T00:13:24.529 INFO:teuthology.orchestra.run.smithi100.stdout:6 2024-04-01T00:13:24.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 msgr2=0x7f9a6c0b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c400 tx=0x7f9a6000c8c0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 shutdown_connections 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0afea0 0x7f9a6c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a6c0ae5a0 0x7f9a6c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 --2- 172.21.15.100:0/4289484004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6c06b600 0x7f9a6c0b3b00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 >> 172.21.15.100:0/4289484004 conn(0x7f9a6c0d0c70 msgr2=0x7f9a6c1601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.529+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 shutdown_connections 2024-04-01T00:13:24.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.530+0000 7f9a73769640 1 -- 172.21.15.100:0/4289484004 wait complete. 2024-04-01T00:13:24.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new f3eb2914-1b89-4a42-8a43-b21819229c26 7 2024-04-01T00:13:24.579 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.574+0000 7fac06c41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-01T00:13:24.597 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.596+0000 7eff3ea40040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-01T00:13:24.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.641+0000 7fc3bdb63640 1 Processor -- start 2024-04-01T00:13:24.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.641+0000 7fc3bdb63640 1 -- start start 2024-04-01T00:13:24.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3bdb63640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3b815e2e0 0x7fc3b815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3bdb63640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b815e6c0 con 0x7fc3b80ceb70 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3bdb63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b815f7f0 con 0x7fc3b80cb2c0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3bdb63640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b80721d0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b6ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc3b80cb2c0 0x7fc3b8072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46058/0 (socket says 172.21.15.100:46058) 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b6ffd640 1 -- 172.21.15.100:0/525575289 learned_addr learned my addr 172.21.15.100:0/525575289 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b77fe640 1 --2- 172.21.15.100:0/525575289 >> v2:172.21.15.64:3300/0 conn(0x7fc3b815e2e0 0x7fc3b815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185997480 0 0) 0x7fc3a8001490 con 0x7fc3b80cb2c0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a0003520 con 0x7fc3b80cb2c0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b77fe640 1 -- 172.21.15.100:0/525575289 >> v1:172.21.15.100:6789/0 conn(0x7fc3b80cb2c0 legacy=0x7fc3b8072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 392850948 0 0) 0x7fc3a0003520 con 0x7fc3b80cb2c0 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b77fe640 1 -- 172.21.15.100:0/525575289 >> v1:172.21.15.64:6789/0 conn(0x7fc3b80ceb70 legacy=0x7fc3b806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.642+0000 7fc3b77fe640 1 -- 172.21.15.100:0/525575289 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b80cb860 con 0x7fc3b815e2e0 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.643+0000 7fc3b77fe640 1 --2- 172.21.15.100:0/525575289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b815e2e0 0x7fc3b815f2b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009720 tx=0x7fc3ac02b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c38ef592af2161a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.643+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac037070 con 0x7fc3b815e2e0 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.643+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac030510 con 0x7fc3b815e2e0 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.643+0000 7fc3b67fc640 1 -- 172.21.15.100:0/525575289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8001bc0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 -- 172.21.15.100:0/525575289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b815e2e0 msgr2=0x7fc3b815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/525575289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b815e2e0 0x7fc3b815f2b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009720 tx=0x7fc3ac02b620 comp rx=0 tx=0).stop 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 -- 172.21.15.100:0/525575289 shutdown_connections 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/525575289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b815e2e0 0x7fc3b815f2b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 -- 172.21.15.100:0/525575289 >> 172.21.15.100:0/525575289 conn(0x7fc3b80d0c70 msgr2=0x7fc3b80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 -- 172.21.15.100:0/525575289 shutdown_connections 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.644+0000 7fc3bdb63640 1 -- 172.21.15.100:0/525575289 wait complete. 2024-04-01T00:13:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.645+0000 7fc3bdb63640 1 Processor -- start 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.645+0000 7fc3bdb63640 1 -- start start 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 0x7fc3b80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 0x7fc3b81479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8072d70 con 0x7fc3b80ceb70 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8071e60 con 0x7fc3b80cb2c0 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3bdb63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b8072ef0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 0x7fc3b80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60928/0 (socket says 172.21.15.100:60928) 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 -- 172.21.15.100:0/1775941890 learned_addr learned my addr 172.21.15.100:0/1775941890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-01T00:13:24.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b6ffd640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 0x7fc3b81479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 -- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 msgr2=0x7fc3b80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 0x7fc3b80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 -- 172.21.15.100:0/1775941890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 msgr2=0x7fc3b81479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 0x7fc3b81479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b7fff640 1 -- 172.21.15.100:0/1775941890 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b80b63c0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b77fe640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 0x7fc3b80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.646+0000 7fc3b6ffd640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 0x7fc3b81479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc3b7fff640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c710 tx=0x7fc3b000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017070 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b0005480 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b80b66a0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000a860 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b80b3f80 con 0x7fc3b815e2e0 2024-04-01T00:13:24.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3b001a070 con 0x7fc3b815e2e0 2024-04-01T00:13:24.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.647+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc3b0003c30 con 0x7fc3b815e2e0 2024-04-01T00:13:24.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.648+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b80688b0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.650+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc3b0021020 con 0x7fc3b815e2e0 2024-04-01T00:13:24.715 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.715+0000 7fac06c41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-01T00:13:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.721+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f3eb2914-1b89-4a42-8a43-b21819229c26", "id": 7} v 0) -- 0x7fc3b80caf00 con 0x7fc3b815e2e0 2024-04-01T00:13:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.727+0000 7fc39ffff640 1 -- 172.21.15.100:0/1775941890 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3eb2914-1b89-4a42-8a43-b21819229c26", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc3b815e6c0 con 0x7fc3b815e2e0 2024-04-01T00:13:24.728 INFO:teuthology.orchestra.run.smithi100.stdout:7 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 msgr2=0x7fc3b8147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c710 tx=0x7fc3b000cbd0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 shutdown_connections 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3b815e2e0 0x7fc3b8147ee0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b80ceb70 0x7fc3b81479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 --2- 172.21.15.100:0/1775941890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3b80cb2c0 0x7fc3b80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 >> 172.21.15.100:0/1775941890 conn(0x7fc3b80d0c70 msgr2=0x7fc3b806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 shutdown_connections 2024-04-01T00:13:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:13:24.728+0000 7fc3bdb63640 1 -- 172.21.15.100:0/1775941890 wait complete. 2024-04-01T00:13:24.736 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.735+0000 7eff3ea40040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-01T00:13:24.740 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-01T00:13:24.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-01T00:13:24.742 INFO:tasks.ceph.osd.0:Started 2024-04-01T00:13:24.742 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-01T00:13:24.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-01T00:13:24.744 INFO:tasks.ceph.osd.1:Started 2024-04-01T00:13:24.744 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-01T00:13:24.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-01T00:13:24.747 INFO:tasks.ceph.osd.2:Started 2024-04-01T00:13:24.747 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-01T00:13:24.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-01T00:13:24.749 INFO:tasks.ceph.osd.3:Started 2024-04-01T00:13:24.749 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-01T00:13:24.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-01T00:13:24.751 INFO:tasks.ceph.osd.4:Started 2024-04-01T00:13:24.751 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-01T00:13:24.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-01T00:13:24.754 INFO:tasks.ceph.osd.5:Started 2024-04-01T00:13:24.754 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-01T00:13:24.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-01T00:13:24.757 INFO:tasks.ceph.osd.6:Started 2024-04-01T00:13:24.757 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-01T00:13:24.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-01T00:13:24.759 INFO:tasks.ceph.osd.7:Started 2024-04-01T00:13:24.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:24.866 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.865+0000 7fac06c41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-01T00:13:24.870 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.869+0000 7eff3ea40040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-01T00:13:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.892+0000 7ff331823640 1 Processor -- start 2024-04-01T00:13:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.892+0000 7ff331823640 1 -- start start 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff331823640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff32c06b7d0 0x7ff32c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff331823640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c072c40 con 0x7ff32c15fae0 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff331823640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c0723a0 con 0x7ff32c160780 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff32b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff32c15fae0 0x7ff32c06f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46008/0 (socket says 172.21.15.64:46008) 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff32b7fe640 1 -- 172.21.15.64:0/2596185079 learned_addr learned my addr 172.21.15.64:0/2596185079 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0a2090 con 0x7ff32c06b7d0 2024-04-01T00:13:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.893+0000 7ff323fff640 1 --2- 172.21.15.64:0/2596185079 >> v2:172.21.15.100:3300/0 conn(0x7ff32c06b7d0 0x7ff32c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.894+0000 7ff32a7fc640 1 -- 172.21.15.64:0/2596185079 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683145332 0 0) 0x7ff32c072c40 con 0x7ff32c15fae0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.894+0000 7ff32a7fc640 1 -- 172.21.15.64:0/2596185079 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff318003520 con 0x7ff32c15fae0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.894+0000 7ff323fff640 1 -- 172.21.15.64:0/2596185079 >> v1:172.21.15.100:6789/0 conn(0x7ff32c160780 legacy=0x7ff32c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.894+0000 7ff323fff640 1 -- 172.21.15.64:0/2596185079 >> v1:172.21.15.64:6789/0 conn(0x7ff32c15fae0 legacy=0x7ff32c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.894+0000 7ff323fff640 1 -- 172.21.15.64:0/2596185079 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c072680 con 0x7ff32c06b7d0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff323fff640 1 --2- 172.21.15.64:0/2596185079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c06b7d0 0x7ff32c15d8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3140097a0 tx=0x7ff31402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7592a6ab07af6f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff32a7fc640 1 -- 172.21.15.64:0/2596185079 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314034070 con 0x7ff32c06b7d0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff32a7fc640 1 -- 172.21.15.64:0/2596185079 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff314030610 con 0x7ff32c06b7d0 2024-04-01T00:13:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff32a7fc640 1 -- 172.21.15.64:0/2596185079 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3140309e0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c06b7d0 msgr2=0x7ff32c15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.895+0000 7ff331823640 1 --2- 172.21.15.64:0/2596185079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c06b7d0 0x7ff32c15d8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3140097a0 tx=0x7ff31402b960 comp rx=0 tx=0).stop 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.896+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 shutdown_connections 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.896+0000 7ff331823640 1 --2- 172.21.15.64:0/2596185079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c06b7d0 0x7ff32c15d8e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.896+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 >> 172.21.15.64:0/2596185079 conn(0x7ff32c0d0be0 msgr2=0x7ff32c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.896+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 shutdown_connections 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.896+0000 7ff331823640 1 -- 172.21.15.64:0/2596185079 wait complete. 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.897+0000 7ff331823640 1 Processor -- start 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.897+0000 7ff331823640 1 -- start start 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff32c15fae0 0x7ff32c0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c160780 0x7ff32c0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c071ca0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c06f1f0 con 0x7ff32c160780 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff331823640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c071e20 con 0x7ff32c15fae0 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41218/0 (socket says 172.21.15.64:41218) 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 -- 172.21.15.64:0/3831270289 learned_addr learned my addr 172.21.15.64:0/3831270289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff323fff640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff32c15fae0 0x7ff32c0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32b7fe640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c160780 0x7ff32c0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 -- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff32c15fae0 msgr2=0x7ff32c0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff32c15fae0 0x7ff32c0b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 -- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c160780 msgr2=0x7ff32c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c160780 0x7ff32c0b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.898+0000 7ff32affd640 1 -- 172.21.15.64:0/3831270289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c157d50 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff32affd640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c400 tx=0x7ff31c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c017070 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c0052d0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c0b64f0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c003830 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c0c4dc0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff31c01a070 con 0x7ff32c06b7d0 2024-04-01T00:13:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.899+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff31c011590 con 0x7ff32c06b7d0 2024-04-01T00:13:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.900+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c0c48f0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.902+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff31c021020 con 0x7ff32c06b7d0 2024-04-01T00:13:24.915 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:24.914+0000 7fac06c41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-01T00:13:24.917 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:24.916+0000 7eff3ea40040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-01T00:13:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.964+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff32c0688b0 con 0x7ff32c06b7d0 2024-04-01T00:13:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff328ff9640 1 -- 172.21.15.64:0/3831270289 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff31c011b80 con 0x7ff32c06b7d0 2024-04-01T00:13:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:24.965 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:24.723519+0000","last_up_change":"0.000000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:24.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 msgr2=0x7ff32c1f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:24.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c400 tx=0x7ff31c00c8c0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 shutdown_connections 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32c160780 0x7ff32c0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff32c15fae0 0x7ff32c0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 --2- 172.21.15.64:0/3831270289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c06b7d0 0x7ff32c1f1f70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 >> 172.21.15.64:0/3831270289 conn(0x7ff32c0d0be0 msgr2=0x7ff32c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 shutdown_connections 2024-04-01T00:13:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:24.965+0000 7ff331823640 1 -- 172.21.15.64:0/3831270289 wait complete. 2024-04-01T00:13:24.977 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-01T00:13:24.978 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-01T00:13:25.029 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:25.026+0000 7eff3ea40040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-01T00:13:25.033 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:25.032+0000 7fac06c41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-01T00:13:25.122 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:25.122+0000 7eff3ea40040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-01T00:13:25.132 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:25.131+0000 7fac06c41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-01T00:13:25.277 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:25.276+0000 7eff3ea40040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-01T00:13:25.292 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:25.291+0000 7fac06c41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-01T00:13:25.319 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:25.318+0000 7eff3ea40040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-01T00:13:25.336 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:25.335+0000 7fac06c41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-01T00:13:25.347 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T00:13:25.346+0000 7fd226ad3640 -1 Falling back to public interface 2024-04-01T00:13:25.349 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T00:13:25.348+0000 7f3993726640 -1 Falling back to public interface 2024-04-01T00:13:25.357 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T00:13:25.356+0000 7f454c815640 -1 Falling back to public interface 2024-04-01T00:13:25.364 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T00:13:25.363+0000 7f4e2061d640 -1 Falling back to public interface 2024-04-01T00:13:25.365 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T00:13:25.364+0000 7f11ffcb1640 -1 Falling back to public interface 2024-04-01T00:13:25.372 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T00:13:25.371+0000 7f1af2813640 -1 Falling back to public interface 2024-04-01T00:13:25.377 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T00:13:25.377+0000 7ff29ef8c640 -1 Falling back to public interface 2024-04-01T00:13:25.380 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T00:13:25.379+0000 7f70e9e13640 -1 Falling back to public interface 2024-04-01T00:13:25.476 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:25.475+0000 7eff3ea40040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-01T00:13:25.502 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:25.501+0000 7fac06c41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-01T00:13:25.783 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:25.875 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 Processor -- start 2024-04-01T00:13:25.875 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 -- start start 2024-04-01T00:13:25.876 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6d14072a30 0x7f6d1415e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:25.876 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d140b17e0 0x7f6d1415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1415eb10 con 0x7f6d14072e10 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f6d140cb960 con 0x7f6d14072a30 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d1a181640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1402bff0 con 0x7f6d140b17e0 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.874+0000 7f6d137fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6d14072a30 0x7f6d1415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6d14072a30 0x7f6d1415e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41232/0 (socket says 172.21.15.64:41232) 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 -- 172.21.15.64:0/1359207692 learned_addr learned my addr 172.21.15.64:0/1359207692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d12ffd640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d140b17e0 0x7f6d1415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 -- 172.21.15.64:0/1359207692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d140b17e0 msgr2=0x7f6d1415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d140b17e0 0x7f6d1415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 -- 172.21.15.64:0/1359207692 >> v1:172.21.15.100:6789/0 conn(0x7f6d14072e10 legacy=0x7f6d14162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.875+0000 7f6d137fe640 1 -- 172.21.15.64:0/1359207692 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140d0fc0 con 0x7f6d14072a30 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d137fe640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072a30 0x7f6d1415e090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6d040098a0 tx=0x7f6d0402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19960dd4ae745a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d127fc640 1 -- 172.21.15.64:0/1359207692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402f830 con 0x7f6d14072a30 2024-04-01T00:13:25.877 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d127fc640 1 -- 172.21.15.64:0/1359207692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0402f9d0 con 0x7f6d14072a30 2024-04-01T00:13:25.878 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d127fc640 1 -- 172.21.15.64:0/1359207692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402fda0 con 0x7f6d14072a30 2024-04-01T00:13:25.880 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 -- 172.21.15.64:0/1359207692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072a30 msgr2=0x7f6d1415e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:25.880 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072a30 0x7f6d1415e090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6d040098a0 tx=0x7f6d0402ba10 comp rx=0 tx=0).stop 2024-04-01T00:13:25.880 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 -- 172.21.15.64:0/1359207692 shutdown_connections 2024-04-01T00:13:25.880 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d140b17e0 0x7f6d1415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.880 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1359207692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072a30 0x7f6d1415e090 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 -- 172.21.15.64:0/1359207692 >> 172.21.15.64:0/1359207692 conn(0x7f6d140d0300 msgr2=0x7f6d140d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 -- 172.21.15.64:0/1359207692 shutdown_connections 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.876+0000 7f6d1a181640 1 -- 172.21.15.64:0/1359207692 wait complete. 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.877+0000 7f6d1a181640 1 Processor -- start 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 -- start start 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d14072a30 0x7f6d1415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d140b17e0 0x7f6d14160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140ce060 con 0x7f6d14072e10 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140cca80 con 0x7f6d140b17e0 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d1a181640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1402c4f0 con 0x7f6d14072a30 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41240/0 (socket says 172.21.15.64:41240) 2024-04-01T00:13:25.881 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d12ffd640 1 -- 172.21.15.64:0/1454854158 learned_addr learned my addr 172.21.15.64:0/1454854158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.878+0000 7f6d137fe640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d14072a30 0x7f6d1415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d13fff640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d140b17e0 0x7f6d14160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 -- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d14072a30 msgr2=0x7f6d1415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d14072a30 0x7f6d1415af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 -- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d140b17e0 msgr2=0x7f6d14160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d140b17e0 0x7f6d14160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 -- 172.21.15.64:0/1454854158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141f5310 con 0x7f6d14072e10 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d12ffd640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9b0 tx=0x7f6cfc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc017070 con 0x7f6d14072e10 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00c9a0 con 0x7f6d14072e10 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1415e690 con 0x7f6d14072e10 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00cd90 con 0x7f6d14072e10 2024-04-01T00:13:25.882 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d140b0270 con 0x7f6d14072e10 2024-04-01T00:13:25.883 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6cfc01a070 con 0x7f6d14072e10 2024-04-01T00:13:25.883 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.879+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6cfc006660 con 0x7f6d14072e10 2024-04-01T00:13:25.883 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.880+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d14069070 con 0x7f6d14072e10 2024-04-01T00:13:25.883 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.883+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6d141f5310 con 0x7f6d14072e10 2024-04-01T00:13:25.947 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.946+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d140b6280 con 0x7f6d14072e10 2024-04-01T00:13:25.947 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.946+0000 7f6d0bfff640 1 -- 172.21.15.64:0/1454854158 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6cfc005990 con 0x7f6d14072e10 2024-04-01T00:13:25.947 INFO:teuthology.misc.health.smithi064.stdout: 2024-04-01T00:13:25.948 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:24.723519+0000","last_up_change":"0.000000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 msgr2=0x7f6d141f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9b0 tx=0x7f6cfc00ee70 comp rx=0 tx=0).stop 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 shutdown_connections 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d140b17e0 0x7f6d14160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d14072e10 0x7f6d141f4dd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 --2- 172.21.15.64:0/1454854158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d14072a30 0x7f6d1415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.947+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 >> 172.21.15.64:0/1454854158 conn(0x7f6d140d0300 msgr2=0x7f6d1406eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.948+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 shutdown_connections 2024-04-01T00:13:25.949 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:25.948+0000 7f6d1a181640 1 -- 172.21.15.64:0/1454854158 wait complete. 2024-04-01T00:13:25.958 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-01T00:13:26.291 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:26.290+0000 7eff3ea40040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-01T00:13:26.348 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:26.347+0000 7fac06c41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-01T00:13:26.569 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:26.568+0000 7eff3ea40040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-01T00:13:26.620 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:26.619+0000 7eff3ea40040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-01T00:13:26.636 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:26.635+0000 7fac06c41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-01T00:13:26.689 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:26.688+0000 7fac06c41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-01T00:13:26.735 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:26.734+0000 7eff3ea40040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-01T00:13:26.810 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-04-01T00:13:26.809+0000 7fac06c41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-01T00:13:27.748 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:27.747+0000 7efedf67d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-01T00:13:27.749 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-01T00:13:27.748+0000 7efedf67d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-01T00:13:28.833 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T00:13:28.832+0000 7f454c815640 -1 osd.3 0 log_to_monitors true 2024-04-01T00:13:28.835 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T00:13:28.835+0000 7f3993726640 -1 osd.5 0 log_to_monitors true 2024-04-01T00:13:28.842 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T00:13:28.842+0000 7fd226ad3640 -1 osd.0 0 log_to_monitors true 2024-04-01T00:13:28.870 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T00:13:28.869+0000 7ff29ef8c640 -1 osd.2 0 log_to_monitors true 2024-04-01T00:13:28.878 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T00:13:28.878+0000 7f11ffcb1640 -1 osd.1 0 log_to_monitors true 2024-04-01T00:13:28.884 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T00:13:28.883+0000 7f4e2061d640 -1 osd.6 0 log_to_monitors true 2024-04-01T00:13:28.900 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T00:13:28.899+0000 7f1af2813640 -1 osd.4 0 log_to_monitors true 2024-04-01T00:13:28.901 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T00:13:28.900+0000 7f70e9e13640 -1 osd.7 0 log_to_monitors true 2024-04-01T00:13:32.766 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:32.822 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T00:13:32.821+0000 7f398ee99640 -1 osd.5 0 waiting for initial osdmap 2024-04-01T00:13:32.832 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T00:13:32.832+0000 7f39874cb640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.839 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T00:13:32.839+0000 7f1aedf86640 -1 osd.4 0 waiting for initial osdmap 2024-04-01T00:13:32.860 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T00:13:32.859+0000 7f1ae65b8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.860 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T00:13:32.859+0000 7f70e5004640 -1 osd.7 0 waiting for initial osdmap 2024-04-01T00:13:32.860 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T00:13:32.860+0000 7fd222246640 -1 osd.0 0 waiting for initial osdmap 2024-04-01T00:13:32.865 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T00:13:32.864+0000 7f4e1cf02640 -1 osd.6 0 waiting for initial osdmap 2024-04-01T00:13:32.869 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T00:13:32.869+0000 7f70dd636640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.871 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T00:13:32.871+0000 7ff29a6ff640 -1 osd.2 0 waiting for initial osdmap 2024-04-01T00:13:32.872 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T00:13:32.872+0000 7fd21a878640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.879 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T00:13:32.879+0000 7f4e13d0d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.883 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T00:13:32.883+0000 7ff292d31640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.901 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.900+0000 7f422a116640 1 Processor -- start 2024-04-01T00:13:32.901 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T00:13:32.900+0000 7f454919a640 -1 osd.3 0 waiting for initial osdmap 2024-04-01T00:13:32.902 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.900+0000 7f422a116640 1 -- start start 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f422a116640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f422a116640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 0x7f42240d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f422a116640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42240b57d0 con 0x7f42240d0a80 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f422a116640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f42240cfdf0 con 0x7f42240c73a0 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f422a116640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f42240cef90 con 0x7f42240d0e60 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f42237fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f42240d0a80 0x7f42240d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:42596/0 (socket says 172.21.15.64:42596) 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f42237fe640 1 -- 172.21.15.64:0/1050220237 learned_addr learned my addr 172.21.15.64:0/1050220237 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:40760/0 (socket says 172.21.15.64:40760) 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4223fff640 1 --2- 172.21.15.64:0/1050220237 >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 0x7f42240d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 -- 172.21.15.64:0/1050220237 >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 msgr2=0x7f42240d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:32.903 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 --2- 172.21.15.64:0/1050220237 >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 0x7f42240d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.904 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 -- 172.21.15.64:0/1050220237 >> v1:172.21.15.100:6789/0 conn(0x7f42240d0a80 legacy=0x7f42240d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:32.904 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4222ffd640 1 -- 172.21.15.64:0/1050220237 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240cf1e0 con 0x7f42240d0e60 2024-04-01T00:13:32.904 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.901+0000 7f4223fff640 1 --2- 172.21.15.64:0/1050220237 >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 0x7f42240d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:32.904 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.903+0000 7f4222ffd640 1 --2- 172.21.15.64:0/1050220237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42180097a0 tx=0x7f421802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe392a563d345769 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:32.909 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.908+0000 7f42227fc640 1 -- 172.21.15.64:0/1050220237 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218035020 con 0x7f42240d0e60 2024-04-01T00:13:32.909 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.908+0000 7f42227fc640 1 -- 172.21.15.64:0/1050220237 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4218031830 con 0x7f42240d0e60 2024-04-01T00:13:32.909 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.908+0000 7f42227fc640 1 -- 172.21.15.64:0/1050220237 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218031c00 con 0x7f42240d0e60 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 -- 172.21.15.64:0/1050220237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 msgr2=0x7f42240cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 --2- 172.21.15.64:0/1050220237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42180097a0 tx=0x7f421802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 -- 172.21.15.64:0/1050220237 shutdown_connections 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 --2- 172.21.15.64:0/1050220237 >> v2:172.21.15.100:3300/0 conn(0x7f42240c73a0 0x7f42240d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 --2- 172.21.15.64:0/1050220237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240d0e60 0x7f42240cea50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 -- 172.21.15.64:0/1050220237 >> 172.21.15.64:0/1050220237 conn(0x7f422409d1f0 msgr2=0x7f42240d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 -- 172.21.15.64:0/1050220237 shutdown_connections 2024-04-01T00:13:32.910 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.909+0000 7f422a116640 1 -- 172.21.15.64:0/1050220237 wait complete. 2024-04-01T00:13:32.911 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.910+0000 7f422a116640 1 Processor -- start 2024-04-01T00:13:32.911 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.910+0000 7f422a116640 1 -- start start 2024-04-01T00:13:32.912 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240c73a0 0x7f42240c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.912 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.912 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 0x7f42240c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f42240cfa80 con 0x7f42240d0a80 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42240cf7a0 con 0x7f42240d0e60 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f422a116640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f42240d2a90 con 0x7f42240c73a0 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f4222ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f4222ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41298/0 (socket says 172.21.15.64:41298) 2024-04-01T00:13:32.913 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f4222ffd640 1 -- 172.21.15.64:0/2716334687 learned_addr learned my addr 172.21.15.64:0/2716334687 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:32.913 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f11fcc5c640 -1 osd.1 0 waiting for initial osdmap 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f4223fff640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 0x7f42240c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.911+0000 7f42237fe640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240c73a0 0x7f42240c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 -- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240c73a0 msgr2=0x7f42240c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240c73a0 0x7f42240c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 -- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 msgr2=0x7f42240c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 0x7f42240c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 -- 172.21.15.64:0/2716334687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240c8710 con 0x7f42240d0a80 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4223fff640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 0x7f42240c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:32.915 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f4222ffd640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f421802bee0 tx=0x7f42180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218041070 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421803b860 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42240c98d0 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218035040 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.912+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42240c9e90 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.914+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 90100+0+0 (secure 0 0 0) 0x7f4218038020 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.914+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4224069070 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.914+0000 7f420bfff640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 0x7f421006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.914+0000 7f42237fe640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 0x7f421006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.914+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f42180ab170 con 0x7f42240d0a80 2024-04-01T00:13:32.916 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.915+0000 7f42237fe640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 0x7f421006ddc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f60 tx=0x7f4214009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:32.918 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T00:13:32.918+0000 7f453ff94640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:32.920 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:32.920+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f421807a5a0 con 0x7f42240d0a80 2024-04-01T00:13:32.926 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T00:13:32.926+0000 7f11f3a56640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-01T00:13:33.100 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.099+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42240d35f0 con 0x7f42240d0a80 2024-04-01T00:13:33.100 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.100+0000 7f420bfff640 1 -- 172.21.15.64:0/2716334687 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f421807e640 con 0x7f42240d0a80 2024-04-01T00:13:33.101 INFO:teuthology.misc.health.smithi064.stdout: 2024-04-01T00:13:33.101 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":12,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:30.753353+0000","last_up_change":"0.000000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 msgr2=0x7f421006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 0x7f421006ddc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f60 tx=0x7f4214009040 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 msgr2=0x7f42241fb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f421802bee0 tx=0x7f42180046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 shutdown_connections 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f421006b900 0x7f421006ddc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42240d0e60 0x7f42240c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42240d0a80 0x7f42241fb750 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 --2- 172.21.15.64:0/2716334687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42240c73a0 0x7f42240c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:33.103 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 >> 172.21.15.64:0/2716334687 conn(0x7f422409d1f0 msgr2=0x7f42240b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:33.104 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 shutdown_connections 2024-04-01T00:13:33.104 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:33.102+0000 7f422a116640 1 -- 172.21.15.64:0/2716334687 wait complete. 2024-04-01T00:13:33.113 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-01T00:13:39.920 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:40.022 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.021+0000 7fb1684f4640 1 Processor -- start 2024-04-01T00:13:40.022 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.021+0000 7fb1684f4640 1 -- start start 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb160161b10 0x7fb1600d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 0x7fb1600b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fb1600b3800 con 0x7fb160161b10 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb1600d27b0 con 0x7fb1600d2240 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb1684f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb1600d2a90 con 0x7fb160161730 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb165a8f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb160161b10 0x7fb1600d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb165a8f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb160161b10 0x7fb1600d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55648/0 (socket says 172.21.15.64:55648) 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb165a8f640 1 -- 172.21.15.64:0/3654122149 learned_addr learned my addr 172.21.15.64:0/3654122149 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:40.024 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb166290640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.022+0000 7fb166a91640 1 --2- 172.21.15.64:0/3654122149 >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 0x7fb1600b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 -- 172.21.15.64:0/3654122149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 msgr2=0x7fb1600d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 -- 172.21.15.64:0/3654122149 >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 msgr2=0x7fb1600b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 --2- 172.21.15.64:0/3654122149 >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 0x7fb1600b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 -- 172.21.15.64:0/3654122149 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1600d2d70 con 0x7fb160161b10 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb166a91640 1 --2- 172.21.15.64:0/3654122149 >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 0x7fb1600b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:40.025 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.023+0000 7fb165a8f640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb160161b10 0x7fb1600d0ce0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb16009d8d0 tx=0x7fb15002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3a5ad4855b292 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.026 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb16528e640 1 -- 172.21.15.64:0/3654122149 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150035020 con 0x7fb160161b10 2024-04-01T00:13:40.026 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb16528e640 1 -- 172.21.15.64:0/3654122149 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb150031870 con 0x7fb160161b10 2024-04-01T00:13:40.026 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb16528e640 1 -- 172.21.15.64:0/3654122149 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150031c40 con 0x7fb160161b10 2024-04-01T00:13:40.026 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 -- 172.21.15.64:0/3654122149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb160161b10 msgr2=0x7fb1600d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb160161b10 0x7fb1600d0ce0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb16009d8d0 tx=0x7fb15002b910 comp rx=0 tx=0).stop 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 -- 172.21.15.64:0/3654122149 shutdown_connections 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 --2- 172.21.15.64:0/3654122149 >> v2:172.21.15.100:3300/0 conn(0x7fb1600d2240 0x7fb1600b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb160161b10 0x7fb1600d0ce0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 --2- 172.21.15.64:0/3654122149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 -- 172.21.15.64:0/3654122149 >> 172.21.15.64:0/3654122149 conn(0x7fb16009d1d0 msgr2=0x7fb1600d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 -- 172.21.15.64:0/3654122149 shutdown_connections 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.024+0000 7fb1684f4640 1 -- 172.21.15.64:0/3654122149 wait complete. 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 Processor -- start 2024-04-01T00:13:40.027 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 -- start start 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 0x7fb1600c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1600b4680 con 0x7fb1600d2240 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1600b4500 con 0x7fb160161b10 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb1684f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb1600b4800 con 0x7fb160161730 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb166290640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb166290640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55658/0 (socket says 172.21.15.64:55658) 2024-04-01T00:13:40.030 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.026+0000 7fb166290640 1 -- 172.21.15.64:0/1005744442 learned_addr learned my addr 172.21.15.64:0/1005744442 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb165a8f640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166a91640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 0x7fb1600c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 -- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 msgr2=0x7fb1600c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 -- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 msgr2=0x7fb1600c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 0x7fb1600c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 -- 172.21.15.64:0/1005744442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1600c9770 con 0x7fb1600d2240 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166a91640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 0x7fb1600c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb166290640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9b0 tx=0x7fb15c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c017070 con 0x7fb1600d2240 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c00c9a0 con 0x7fb1600d2240 2024-04-01T00:13:40.031 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cd90 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1600c9a50 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.027+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1600ca010 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.028+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb160069070 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.029+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb15c01a070 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.029+0000 7fb156ffd640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 0x7fb14406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.029+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb15c08a8e0 con 0x7fb1600d2240 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.029+0000 7fb165a8f640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 0x7fb14406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.032 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.030+0000 7fb165a8f640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 0x7fb14406de50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb150036000 tx=0x7fb15003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.034 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.034+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb15c059840 con 0x7fb1600d2240 2024-04-01T00:13:40.217 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.216+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1600b6360 con 0x7fb1600d2240 2024-04-01T00:13:40.217 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.217+0000 7fb156ffd640 1 -- 172.21.15.64:0/1005744442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12860 (secure 0 0 0) 0x7fb15c059140 con 0x7fb1600d2240 2024-04-01T00:13:40.218 INFO:teuthology.misc.health.smithi064.stdout: 2024-04-01T00:13:40.218 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":16,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:36.913233+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:40.219 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.218+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 msgr2=0x7fb14406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.218+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 0x7fb14406de50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb150036000 tx=0x7fb15003c040 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.218+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 msgr2=0x7fb1601542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.218+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9b0 tx=0x7fb15c00ee70 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 shutdown_connections 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb14406b990 0x7fb14406de50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb160161b10 0x7fb1600c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb160161730 0x7fb1600c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 --2- 172.21.15.64:0/1005744442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1600d2240 0x7fb1601542b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 >> 172.21.15.64:0/1005744442 conn(0x7fb16009d1d0 msgr2=0x7fb16006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 shutdown_connections 2024-04-01T00:13:40.220 INFO:teuthology.misc.health.smithi064.stderr:2024-04-01T00:13:40.219+0000 7fb1684f4640 1 -- 172.21.15.64:0/1005744442 wait complete. 2024-04-01T00:13:40.229 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-01T00:13:40.230 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-01T00:13:40.230 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-01T00:13:40.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-01T00:13:40.232 INFO:tasks.ceph.mds.a:Started 2024-04-01T00:13:40.232 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-01T00:13:40.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-01T00:13:40.234 INFO:tasks.ceph.mds.c:Started 2024-04-01T00:13:40.234 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-01T00:13:40.235 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-01T00:13:40.237 INFO:tasks.ceph.mds.b:Started 2024-04-01T00:13:40.237 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-01T00:13:40.237 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-01T00:13:40.239 INFO:tasks.ceph.mds.d:Started 2024-04-01T00:13:40.240 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-01T00:13:40.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:40.309 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-04-01T00:13:40.309 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-04-01T00:13:40.324 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-01T00:13:40.325 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-01T00:13:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 Processor -- start 2024-04-01T00:13:40.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 -- start start 2024-04-01T00:13:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcba40d1920 0x7fcba40d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 0x7fcba4161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcba40d2780 con 0x7fcba406c3e0 2024-04-01T00:13:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcba40cf3a0 con 0x7fcba40d1920 2024-04-01T00:13:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.348+0000 7fcbaaecd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcba40d2240 con 0x7fcba415e3f0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba946a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcba406c3e0 0x7fcba4162220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55056/0 (socket says 172.21.15.64:55056) 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba946a640 1 -- 172.21.15.64:0/3718761563 learned_addr learned my addr 172.21.15.64:0/3718761563 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba8c69640 1 --2- 172.21.15.64:0/3718761563 >> v2:172.21.15.100:3300/0 conn(0x7fcba40d1920 0x7fcba40d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba3fff640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 0x7fcba4161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841672351 0 0) 0x7fcba40d2780 con 0x7fcba406c3e0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb94003520 con 0x7fcba406c3e0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522472939 0 0) 0x7fcb94003520 con 0x7fcba406c3e0 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba8c69640 1 -- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 msgr2=0x7fcba4161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba8c69640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 0x7fcba4161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba8c69640 1 -- 172.21.15.64:0/3718761563 >> v1:172.21.15.64:6789/0 conn(0x7fcba406c3e0 legacy=0x7fcba4162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba8c69640 1 -- 172.21.15.64:0/3718761563 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba40ce790 con 0x7fcba40d1920 2024-04-01T00:13:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.349+0000 7fcba3fff640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 0x7fcba4161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.350+0000 7fcba8c69640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba40d1920 0x7fcba40d1d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb9802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63678e6139cc793f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.350+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98037070 con 0x7fcba40d1920 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.350+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb98030510 con 0x7fcba40d1920 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.350+0000 7fcba37fe640 1 -- 172.21.15.64:0/3718761563 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0019f0 con 0x7fcba40d1920 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 -- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba40d1920 msgr2=0x7fcba40d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba40d1920 0x7fcba40d1d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb9802b790 comp rx=0 tx=0).stop 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 -- 172.21.15.64:0/3718761563 shutdown_connections 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba415e3f0 0x7fcba4161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/3718761563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba40d1920 0x7fcba40d1d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 -- 172.21.15.64:0/3718761563 >> 172.21.15.64:0/3718761563 conn(0x7fcba409a980 msgr2=0x7fcba409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 -- 172.21.15.64:0/3718761563 shutdown_connections 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.351+0000 7fcbaaecd640 1 -- 172.21.15.64:0/3718761563 wait complete. 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.352+0000 7fcbaaecd640 1 Processor -- start 2024-04-01T00:13:40.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.352+0000 7fcbaaecd640 1 -- start start 2024-04-01T00:13:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 0x7fcba40c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcba40cf030 con 0x7fcba40d1920 2024-04-01T00:13:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcba40ced50 con 0x7fcba406c3e0 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcbaaecd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcba4162930 con 0x7fcba4152050 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:46754/0 (socket says 172.21.15.64:46754) 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55694/0 (socket says 172.21.15.64:55694) 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 -- 172.21.15.64:0/822489549 learned_addr learned my addr 172.21.15.64:0/822489549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba946a640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 0x7fcba40c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 -- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 msgr2=0x7fcba40c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 0x7fcba40c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 -- 172.21.15.64:0/822489549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 msgr2=0x7fcba40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 -- 172.21.15.64:0/822489549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba40c60d0 con 0x7fcba406c3e0 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba3fff640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba946a640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 0x7fcba40c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.353+0000 7fcba8c69640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098c0 tx=0x7fcb980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.354+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98041020 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.354+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9803c870 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.354+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba40c63b0 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.354+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98037040 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.354+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba40c6970 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.355+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba40681a0 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.355+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fcb98044020 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.355+0000 7fcba17fa640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 0x7fcb8806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.356+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcb980aba00 con 0x7fcba406c3e0 2024-04-01T00:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.356+0000 7fcba3fff640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 0x7fcb8806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.356+0000 7fcba3fff640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 0x7fcb8806de80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcba4159970 tx=0x7fcb90009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.360+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fcb9804b070 con 0x7fcba406c3e0 2024-04-01T00:13:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.543+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcba406f530 con 0x7fcba406c3e0 2024-04-01T00:13:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.544+0000 7fcba17fa640 1 -- 172.21.15.64:0/822489549 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12860 (secure 0 0 0) 0x7fcb9807a3a0 con 0x7fcba406c3e0 2024-04-01T00:13:40.545 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:40.546 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:36.913233+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 msgr2=0x7fcb8806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 0x7fcb8806de80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcba4159970 tx=0x7fcb90009190 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 msgr2=0x7fcba415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098c0 tx=0x7fcb980046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 shutdown_connections 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcb8806b9c0 0x7fcb8806de80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcba4152050 0x7fcba40c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba40d1920 0x7fcba40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 --2- 172.21.15.64:0/822489549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcba406c3e0 0x7fcba415ba00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.546+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 >> 172.21.15.64:0/822489549 conn(0x7fcba409a980 msgr2=0x7fcba40b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.547+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 shutdown_connections 2024-04-01T00:13:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.547+0000 7fcbaaecd640 1 -- 172.21.15.64:0/822489549 wait complete. 2024-04-01T00:13:40.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:13:40.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:13:40.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.662+0000 7f6cb700f640 1 Processor -- start 2024-04-01T00:13:40.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.662+0000 7f6cb700f640 1 -- start start 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb700f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb015e7c0 0x7f6cb00d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb700f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb00cee70 con 0x7f6cb00d15e0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb700f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb00cf150 con 0x7f6cb015e3e0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb700f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00cf430 con 0x7f6cb015e7c0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb55ac640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6cb00d15e0 0x7f6cb00d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55066/0 (socket says 172.21.15.64:55066) 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6cb55ac640 1 -- 172.21.15.64:0/3208977434 learned_addr learned my addr 172.21.15.64:0/3208977434 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caffff640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb015e7c0 0x7f6cb00d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349800636 0 0) 0x7f6cb00cee70 con 0x7f6cb00d15e0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c9c003520 con 0x7f6cb00d15e0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4049097563 0 0) 0x7f6c9c003520 con 0x7f6cb00d15e0 2024-04-01T00:13:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb015e7c0 msgr2=0x7f6cb00d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.663+0000 7f6caf7fe640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb015e7c0 0x7f6cb00d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 >> v1:172.21.15.100:6789/0 conn(0x7f6cb015e3e0 legacy=0x7f6cb00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00cf710 con 0x7f6cb00d15e0 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f6c98001670 con 0x7f6cb00d15e0 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 >> v1:172.21.15.64:6789/0 conn(0x7f6cb00d15e0 legacy=0x7f6cb00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c0039e0 0x7f6c9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c9c004390 0x7f6c9c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c005010 0x7f6c9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00cf430 con 0x7f6c9c0039e0 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c00bf30 con 0x7f6c9c005010 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c00c130 con 0x7f6c9c004390 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caffff640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb015e7c0 0x7f6cb00d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c0039e0 0x7f6c9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb55ac640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c005010 0x7f6c9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caffff640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c9c004390 0x7f6c9c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 -- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c9c004390 msgr2=0x7f6c9c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c9c004390 0x7f6c9c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 -- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c005010 msgr2=0x7f6c9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c005010 0x7f6c9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6cb4dab640 1 -- 172.21.15.64:0/3208977434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c003520 con 0x7f6c9c0039e0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.664+0000 7f6caffff640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c9c004390 0x7f6c9c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c0039e0 0x7f6c9c003dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8001860 tx=0x7f6c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3aabedccf454bc server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca802f050 con 0x7f6c9c0039e0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8033650 con 0x7f6c9c0039e0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6caf7fe640 1 -- 172.21.15.64:0/3208977434 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8001430 con 0x7f6c9c0039e0 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6cb700f640 1 -- 172.21.15.64:0/3208977434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c0039e0 msgr2=0x7f6c9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6cb700f640 1 --2- 172.21.15.64:0/3208977434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c0039e0 0x7f6c9c003dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8001860 tx=0x7f6c9c005830 comp rx=0 tx=0).stop 2024-04-01T00:13:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6cb55ac640 1 -- 172.21.15.64:0/3208977434 reap_dead start 2024-04-01T00:13:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.665+0000 7f6cb700f640 1 -- 172.21.15.64:0/3208977434 shutdown_connections 2024-04-01T00:13:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.666+0000 7f6cb700f640 1 -- 172.21.15.64:0/3208977434 >> 172.21.15.64:0/3208977434 conn(0x7f6cb009d210 msgr2=0x7f6cb00d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.666+0000 7f6cb700f640 1 -- 172.21.15.64:0/3208977434 shutdown_connections 2024-04-01T00:13:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.666+0000 7f6cb700f640 1 -- 172.21.15.64:0/3208977434 wait complete. 2024-04-01T00:13:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.667+0000 7f6cb700f640 1 Processor -- start 2024-04-01T00:13:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.667+0000 7f6cb700f640 1 -- start start 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c0039e0 0x7f6cb00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 0x7f6cb01f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00d2b60 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00d29e0 con 0x7f6c9c0039e0 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00d4720 con 0x7f6cb00cde60 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55714/0 (socket says 172.21.15.64:55714) 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 -- 172.21.15.64:0/1223490113 learned_addr learned my addr 172.21.15.64:0/1223490113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c0039e0 0x7f6cb00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6caffff640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 0x7f6cb01f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 -- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 msgr2=0x7f6cb01f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 0x7f6cb01f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 -- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c0039e0 msgr2=0x7f6cb00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c0039e0 0x7f6cb00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 -- 172.21.15.64:0/1223490113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00cbb20 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6caffff640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 0x7f6cb01f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb55ac640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6cb008c1a0 tx=0x7f6c98007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9801a070 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9800f6a0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb00cbe00 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800fa90 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.668+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb00cc3c0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.669+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb00d4bb0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.670+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6c9801d070 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.670+0000 7f6cad7fa640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 0x7f6c9006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.670+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6c980898d0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.670+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 0x7f6c9006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.671+0000 7f6cb4dab640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 0x7f6c9006de50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036000 tx=0x7f6ca8031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.674+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6cb00d4bb0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.854+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6cb00681a0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.854+0000 7f6cad7fa640 1 -- 172.21.15.64:0/1223490113 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f6c980581e0 con 0x7f6cb01f7ce0 2024-04-01T00:13:40.855 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 msgr2=0x7f6c9006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 0x7f6c9006de50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036000 tx=0x7f6ca8031000 comp rx=0 tx=0).stop 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 msgr2=0x7f6cb01f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6cb008c1a0 tx=0x7f6c98007430 comp rx=0 tx=0).stop 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 shutdown_connections 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6c9006b990 0x7f6c9006de50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb01f7ce0 0x7f6cb01f0e40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cb00cde60 0x7f6cb01f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 --2- 172.21.15.64:0/1223490113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c0039e0 0x7f6cb00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 >> 172.21.15.64:0/1223490113 conn(0x7f6cb009d210 msgr2=0x7f6cb015ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 shutdown_connections 2024-04-01T00:13:40.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.856+0000 7f6cb700f640 1 -- 172.21.15.64:0/1223490113 wait complete. 2024-04-01T00:13:40.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:13:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 Processor -- start 2024-04-01T00:13:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 -- start start 2024-04-01T00:13:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04880d4a60 0x7f04880b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04880cfae0 0x7f04881620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04880d2ba0 con 0x7f04881619e0 2024-04-01T00:13:40.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f04880d2e80 con 0x7f04880cfae0 2024-04-01T00:13:40.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048cceb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f04880ceeb0 con 0x7f04880d4a60 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f0486d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04880cfae0 0x7f04881620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04881619e0 0x7f04880cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55070/0 (socket says 172.21.15.64:55070) 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.969+0000 7f048659c640 1 -- 172.21.15.64:0/3869343951 learned_addr learned my addr 172.21.15.64:0/3869343951 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0486d9d640 1 -- 172.21.15.64:0/3869343951 >> v2:172.21.15.100:3300/0 conn(0x7f04880d4a60 msgr2=0x7f04880b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0486d9d640 1 --2- 172.21.15.64:0/3869343951 >> v2:172.21.15.100:3300/0 conn(0x7f04880d4a60 0x7f04880b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0486d9d640 1 -- 172.21.15.64:0/3869343951 >> v1:172.21.15.64:6789/0 conn(0x7f04881619e0 legacy=0x7f04880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0486d9d640 1 -- 172.21.15.64:0/3869343951 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488161dc0 con 0x7f04880cfae0 2024-04-01T00:13:40.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0485d9b640 1 -- 172.21.15.64:0/3869343951 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830496601 0 0) 0x7f04880d2ba0 con 0x7f04881619e0 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.970+0000 7f0486d9d640 1 --2- 172.21.15.64:0/3869343951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881620a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f048808c0a0 tx=0x7f048002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac78d353af2439f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f0485d9b640 1 -- 172.21.15.64:0/3869343951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480039070 con 0x7f04880cfae0 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f0485d9b640 1 -- 172.21.15.64:0/3869343951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048002cbf0 con 0x7f04880cfae0 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f0485d9b640 1 -- 172.21.15.64:0/3869343951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480034440 con 0x7f04880cfae0 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 -- 172.21.15.64:0/3869343951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 msgr2=0x7f04881620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 --2- 172.21.15.64:0/3869343951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881620a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f048808c0a0 tx=0x7f048002be40 comp rx=0 tx=0).stop 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 -- 172.21.15.64:0/3869343951 shutdown_connections 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 --2- 172.21.15.64:0/3869343951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881620a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 --2- 172.21.15.64:0/3869343951 >> v2:172.21.15.100:3300/0 conn(0x7f04880d4a60 0x7f04880b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 -- 172.21.15.64:0/3869343951 >> 172.21.15.64:0/3869343951 conn(0x7f048809d290 msgr2=0x7f04880d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:40.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 -- 172.21.15.64:0/3869343951 shutdown_connections 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.971+0000 7f048cceb640 1 -- 172.21.15.64:0/3869343951 wait complete. 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.972+0000 7f048cceb640 1 Processor -- start 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 -- start start 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 0x7f04880b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 0x7f04880b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04880ceb10 con 0x7f04880cfae0 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04880b31b0 con 0x7f04880d4a60 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048cceb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f04880cec90 con 0x7f04881619e0 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55732/0 (socket says 172.21.15.64:55732) 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f048659c640 1 -- 172.21.15.64:0/1098047595 learned_addr learned my addr 172.21.15.64:0/1098047595 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.973+0000 7f0486d9d640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 0x7f04880b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 -- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 msgr2=0x7f04880b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f047ffff640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 0x7f04880b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 0x7f04880b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 -- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 msgr2=0x7f04880b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 0x7f04880b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 -- 172.21.15.64:0/1098047595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04880cbe70 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f0486d9d640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 0x7f04880b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f047ffff640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 0x7f04880b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048659c640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f047800c460 tx=0x7f047800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478018070 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0478007500 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04880cc150 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478004a20 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.974+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04880cc710 con 0x7f04880cfae0 2024-04-01T00:13:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.975+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04880681a0 con 0x7f04880cfae0 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.975+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f047801b070 con 0x7f04880cfae0 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.976+0000 7f047effd640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 0x7f046806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.976+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0478089730 con 0x7f04880cfae0 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.976+0000 7f047ffff640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 0x7f046806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.976+0000 7f047ffff640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 0x7f046806de50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0470006fd0 tx=0x7f0470008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:40.980+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0478058720 con 0x7f04880cfae0 2024-04-01T00:13:41.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.201+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f048809d730 con 0x7f04880cfae0 2024-04-01T00:13:41.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.201+0000 7f047effd640 1 -- 172.21.15.64:0/1098047595 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f0478058020 con 0x7f04880cfae0 2024-04-01T00:13:41.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:41.202 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-01T00:13:41.203 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-04-01T00:13:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 msgr2=0x7f046806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 0x7f046806de50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0470006fd0 tx=0x7f0470008040 comp rx=0 tx=0).stop 2024-04-01T00:13:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 msgr2=0x7f04881f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f047800c460 tx=0x7f047800c920 comp rx=0 tx=0).stop 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 shutdown_connections 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f046806b990 0x7f046806de50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881619e0 0x7f04880b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04880d4a60 0x7f04880b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 --2- 172.21.15.64:0/1098047595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04880cfae0 0x7f04881f34d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 >> 172.21.15.64:0/1098047595 conn(0x7f048809d290 msgr2=0x7f048806bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 shutdown_connections 2024-04-01T00:13:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:41.203+0000 7f048cceb640 1 -- 172.21.15.64:0/1098047595 wait complete. 2024-04-01T00:13:43.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:13:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 Processor -- start 2024-04-01T00:13:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 -- start start 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f881619e0 0x7f3f880d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cfae0 0x7f3f881620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f880b3cb0 con 0x7f3f880b3820 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3f880d2ba0 con 0x7f3f881619e0 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.330+0000 7f3f8e156640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f880d2e80 con 0x7f3f880cfae0 2024-04-01T00:13:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f881619e0 0x7f3f880d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f86ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3f880b3820 0x7f3f880cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55080/0 (socket says 172.21.15.64:55080) 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f881619e0 0x7f3f880d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55738/0 (socket says 172.21.15.64:55738) 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f86ffd640 1 -- 172.21.15.64:0/643155804 learned_addr learned my addr 172.21.15.64:0/643155804 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f87fff640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cfae0 0x7f3f881620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 -- 172.21.15.64:0/643155804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cfae0 msgr2=0x7f3f881620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cfae0 0x7f3f881620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 -- 172.21.15.64:0/643155804 >> v1:172.21.15.64:6789/0 conn(0x7f3f880b3820 legacy=0x7f3f880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f877fe640 1 -- 172.21.15.64:0/643155804 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f880ceeb0 con 0x7f3f881619e0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.331+0000 7f3f867fc640 1 -- 172.21.15.64:0/643155804 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750515581 0 0) 0x7f3f880b3cb0 con 0x7f3f880b3820 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f877fe640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f880d4a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097a0 tx=0x7f3f7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ded157ed8be5de server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f867fc640 1 -- 172.21.15.64:0/643155804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780315a0 con 0x7f3f881619e0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f867fc640 1 -- 172.21.15.64:0/643155804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78037070 con 0x7f3f881619e0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f867fc640 1 -- 172.21.15.64:0/643155804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78031960 con 0x7f3f881619e0 2024-04-01T00:13:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f8e156640 1 -- 172.21.15.64:0/643155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 msgr2=0x7f3f880d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.332+0000 7f3f8e156640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f880d4a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097a0 tx=0x7f3f7802b840 comp rx=0 tx=0).stop 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 -- 172.21.15.64:0/643155804 shutdown_connections 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cfae0 0x7f3f881620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 --2- 172.21.15.64:0/643155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f880d4a60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 -- 172.21.15.64:0/643155804 >> 172.21.15.64:0/643155804 conn(0x7f3f8809d290 msgr2=0x7f3f880d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 -- 172.21.15.64:0/643155804 shutdown_connections 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.333+0000 7f3f8e156640 1 -- 172.21.15.64:0/643155804 wait complete. 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.334+0000 7f3f8e156640 1 Processor -- start 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.334+0000 7f3f8e156640 1 -- start start 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880b3820 0x7f3f880b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 0x7f3f880b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880b31b0 con 0x7f3f881619e0 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880d2890 con 0x7f3f880cfae0 2024-04-01T00:13:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f880ceb10 con 0x7f3f880b3820 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55752/0 (socket says 172.21.15.64:55752) 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 -- 172.21.15.64:0/507256413 learned_addr learned my addr 172.21.15.64:0/507256413 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f877fe640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880b3820 0x7f3f880b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f86ffd640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 0x7f3f880b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 -- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880b3820 msgr2=0x7f3f880b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880b3820 0x7f3f880b9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 -- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 msgr2=0x7f3f880b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 0x7f3f880b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 -- 172.21.15.64:0/507256413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f881f79d0 con 0x7f3f881619e0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f86ffd640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 0x7f3f880b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f87fff640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009dd0 tx=0x7f3f8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80018020 con 0x7f3f881619e0 2024-04-01T00:13:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f80014480 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f881f7cb0 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.335+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80014870 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.336+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f881f8270 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.337+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f880681a0 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.337+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3f80005d10 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.338+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 0x7f3f7406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.338+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3f8008af50 con 0x7f3f881619e0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.338+0000 7f3f877fe640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 0x7f3f7406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.338+0000 7f3f877fe640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 0x7f3f7406de50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f7803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.342+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3f80022070 con 0x7f3f881619e0 2024-04-01T00:13:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.564+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8809d730 con 0x7f3f881619e0 2024-04-01T00:13:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.564+0000 7f3f6ffff640 1 -- 172.21.15.64:0/507256413 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f3f80010040 con 0x7f3f881619e0 2024-04-01T00:13:43.565 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:43.566 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-01T00:13:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-04-01T00:13:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 msgr2=0x7f3f7406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 0x7f3f7406de50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f7803d040 comp rx=0 tx=0).stop 2024-04-01T00:13:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 msgr2=0x7f3f881523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009dd0 tx=0x7f3f8000c490 comp rx=0 tx=0).stop 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 shutdown_connections 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f7406b990 0x7f3f7406de50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f881619e0 0x7f3f881523a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f880cfae0 0x7f3f880b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 --2- 172.21.15.64:0/507256413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880b3820 0x7f3f880b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 >> 172.21.15.64:0/507256413 conn(0x7f3f8809d290 msgr2=0x7f3f880d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 shutdown_connections 2024-04-01T00:13:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.566+0000 7f3f8e156640 1 -- 172.21.15.64:0/507256413 wait complete. 2024-04-01T00:13:43.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.679+0000 7f9757c2a640 1 Processor -- start 2024-04-01T00:13:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 -- start start 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97500d2740 0x7f9750161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97500b8310 0x7f97501619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97500b5070 con 0x7f97500d1030 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f97500d4e60 con 0x7f97500b8310 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.680+0000 7f9757c2a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f97500b5350 con 0x7f97500d2740 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97561c7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97500b8310 0x7f97501619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97551c5640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f97500d1030 0x7f97500ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42832/0 (socket says 172.21.15.64:42832) 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97551c5640 1 -- 172.21.15.64:0/3649041411 learned_addr learned my addr 172.21.15.64:0/3649041411 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97559c6640 1 --2- 172.21.15.64:0/3649041411 >> v2:172.21.15.100:3300/0 conn(0x7f97500d2740 0x7f9750161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97561c7640 1 -- 172.21.15.64:0/3649041411 >> v2:172.21.15.100:3300/0 conn(0x7f97500d2740 msgr2=0x7f9750161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97561c7640 1 --2- 172.21.15.64:0/3649041411 >> v2:172.21.15.100:3300/0 conn(0x7f97500d2740 0x7f9750161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97561c7640 1 -- 172.21.15.64:0/3649041411 >> v1:172.21.15.64:6789/0 conn(0x7f97500d1030 legacy=0x7f97500ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.681+0000 7f97561c7640 1 -- 172.21.15.64:0/3649041411 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500b6de0 con 0x7f97500b8310 2024-04-01T00:13:43.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f97561c7640 1 --2- 172.21.15.64:0/3649041411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500b8310 0x7f97501619c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97500d3e50 tx=0x7f974002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563c37d0bdf97478 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f97549c4640 1 -- 172.21.15.64:0/3649041411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740039070 con 0x7f97500b8310 2024-04-01T00:13:43.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f97549c4640 1 -- 172.21.15.64:0/3649041411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974002cc80 con 0x7f97500b8310 2024-04-01T00:13:43.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f97549c4640 1 -- 172.21.15.64:0/3649041411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740034660 con 0x7f97500b8310 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 -- 172.21.15.64:0/3649041411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500b8310 msgr2=0x7f97501619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3649041411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500b8310 0x7f97501619c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97500d3e50 tx=0x7f974002bed0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 -- 172.21.15.64:0/3649041411 shutdown_connections 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3649041411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500b8310 0x7f97501619c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3649041411 >> v2:172.21.15.100:3300/0 conn(0x7f97500d2740 0x7f9750161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 -- 172.21.15.64:0/3649041411 >> 172.21.15.64:0/3649041411 conn(0x7f97500d3ef0 msgr2=0x7f97500d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 -- 172.21.15.64:0/3649041411 shutdown_connections 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.682+0000 7f9757c2a640 1 -- 172.21.15.64:0/3649041411 wait complete. 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.683+0000 7f9757c2a640 1 Processor -- start 2024-04-01T00:13:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 -- start start 2024-04-01T00:13:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97500b8310 0x7f97501f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97500d2740 0x7f9750153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97500d4b90 con 0x7f97500d1030 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97500d4a10 con 0x7f97500b8310 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f9757c2a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97500d1cd0 con 0x7f97500d2740 2024-04-01T00:13:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f97551c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f97551c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53266/0 (socket says 172.21.15.64:53266) 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.684+0000 7f97551c5640 1 -- 172.21.15.64:0/3709979514 learned_addr learned my addr 172.21.15.64:0/3709979514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97559c6640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97500b8310 0x7f97501f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97561c7640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97500d2740 0x7f9750153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 -- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97500d2740 msgr2=0x7f9750153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97500d2740 0x7f9750153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 -- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97500b8310 msgr2=0x7f97501f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97500b8310 0x7f97501f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 -- 172.21.15.64:0/3709979514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97501f3dd0 con 0x7f97500d1030 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f97551c5640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f974400c400 tx=0x7f974400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744017070 con 0x7f97500d1030 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9744005ce0 con 0x7f97500d1030 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975009e4f0 con 0x7f97500d1030 2024-04-01T00:13:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97440072f0 con 0x7f97500d1030 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.685+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975009eab0 con 0x7f97500d1030 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.686+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500681a0 con 0x7f97500d1030 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.686+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f974401a070 con 0x7f97500d1030 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.687+0000 7f974e7fc640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 0x7f973c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.687+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9744089930 con 0x7f97500d1030 2024-04-01T00:13:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.687+0000 7f97559c6640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 0x7f973c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.687+0000 7f97559c6640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 0x7f973c06de50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9748006fd0 tx=0x7f9748008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.691+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f9744058890 con 0x7f97500d1030 2024-04-01T00:13:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.875+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97500d22b0 con 0x7f97500d1030 2024-04-01T00:13:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.876+0000 7f974e7fc640 1 -- 172.21.15.64:0/3709979514 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12860 (secure 0 0 0) 0x7f9744058190 con 0x7f97500d1030 2024-04-01T00:13:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:43.877 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:36.913233+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 msgr2=0x7f973c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 0x7f973c06de50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9748006fd0 tx=0x7f9748008040 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 msgr2=0x7f975009f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f974400c400 tx=0x7f974400c8c0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 shutdown_connections 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f973c06b990 0x7f973c06de50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97500d2740 0x7f9750153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97500d1030 0x7f975009f590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 --2- 172.21.15.64:0/3709979514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97500b8310 0x7f97501f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 >> 172.21.15.64:0/3709979514 conn(0x7f97500d3ef0 msgr2=0x7f97501608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 shutdown_connections 2024-04-01T00:13:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.878+0000 7f9757c2a640 1 -- 172.21.15.64:0/3709979514 wait complete. 2024-04-01T00:13:43.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:13:43.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:13:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.995+0000 7f6750be2640 1 Processor -- start 2024-04-01T00:13:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.995+0000 7f6750be2640 1 -- start start 2024-04-01T00:13:43.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f6750be2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f674c0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f6750be2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c0cf9f0 con 0x7f674c06b7d0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f6750be2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c0cfcd0 con 0x7f674c0cf4c0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f6750be2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0cffb0 con 0x7f674c1459a0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674ad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f674c06b7d0 0x7f674c06dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42848/0 (socket says 172.21.15.64:42848) 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674ad9d640 1 -- 172.21.15.64:0/4049387666 learned_addr learned my addr 172.21.15.64:0/4049387666 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674a59c640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f674c0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269839857 0 0) 0x7f674c0cf9f0 con 0x7f674c06b7d0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6738003520 con 0x7f674c06b7d0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4135058301 0 0) 0x7f6738003520 con 0x7f674c06b7d0 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 msgr2=0x7f674c0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f674c0c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 >> v1:172.21.15.100:6789/0 conn(0x7f674c0cf4c0 legacy=0x7f674c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c0d0290 con 0x7f674c06b7d0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f67440019f0 con 0x7f674c06b7d0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 >> v1:172.21.15.64:6789/0 conn(0x7f674c06b7d0 legacy=0x7f674c06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f6738003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f6738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0cffb0 con 0x7f67380039e0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f673800ba70 con 0x7f6738004d40 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f673800bc70 con 0x7f674c1459a0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.996+0000 7f674a59c640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f6738003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674ad9d640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f6749d9b640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f6738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 -- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 msgr2=0x7f6738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c1459a0 0x7f6738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 -- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6738004d40 msgr2=0x7f67380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 -- 172.21.15.64:0/4049387666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6738003520 con 0x7f67380039e0 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674a59c640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f6738003e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f673c009a90 tx=0x7f6738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beea3a750ba1d77d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c039070 con 0x7f67380039e0 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c02c680 con 0x7f67380039e0 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.997+0000 7f674959a640 1 -- 172.21.15.64:0/4049387666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c009690 con 0x7f67380039e0 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 -- 172.21.15.64:0/4049387666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 msgr2=0x7f6738003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 --2- 172.21.15.64:0/4049387666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f6738003e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f673c009a90 tx=0x7f6738005830 comp rx=0 tx=0).stop 2024-04-01T00:13:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f674ad9d640 1 -- 172.21.15.64:0/4049387666 reap_dead start 2024-04-01T00:13:44.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 -- 172.21.15.64:0/4049387666 shutdown_connections 2024-04-01T00:13:44.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 -- 172.21.15.64:0/4049387666 >> 172.21.15.64:0/4049387666 conn(0x7f674c0d0bb0 msgr2=0x7f674c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:44.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 -- 172.21.15.64:0/4049387666 shutdown_connections 2024-04-01T00:13:44.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.998+0000 7f6750be2640 1 -- 172.21.15.64:0/4049387666 wait complete. 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.999+0000 7f6750be2640 1 Processor -- start 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:43.999+0000 7f6750be2640 1 -- start start 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f674c0ca030 0x7f674c0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c13f970 0x7f674c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0c6060 con 0x7f67380039e0 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0c5ee0 con 0x7f674c0ca030 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6750be2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0c61e0 con 0x7f674c13f970 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f674a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f674a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53276/0 (socket says 172.21.15.64:53276) 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f674a59c640 1 -- 172.21.15.64:0/4232343331 learned_addr learned my addr 172.21.15.64:0/4232343331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f674ad9d640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c13f970 0x7f674c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.000+0000 7f6749d9b640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f674c0ca030 0x7f674c0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 -- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c13f970 msgr2=0x7f674c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c13f970 0x7f674c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 -- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f674c0ca030 msgr2=0x7f674c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f674c0ca030 0x7f674c0b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 -- 172.21.15.64:0/4232343331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c0c7d20 con 0x7f67380039e0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.001+0000 7f674a59c640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f673c000c00 tx=0x7f673c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.002+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c042020 con 0x7f67380039e0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.002+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c03a040 con 0x7f67380039e0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.002+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c0c8000 con 0x7f67380039e0 2024-04-01T00:13:44.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.002+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c039070 con 0x7f67380039e0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.002+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c0c85c0 con 0x7f67380039e0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.003+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c0681a0 con 0x7f67380039e0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.003+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f673c045020 con 0x7f67380039e0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.004+0000 7f6742ffd640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 0x7f672c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.004+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f673c0ab840 con 0x7f67380039e0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.004+0000 7f6749d9b640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 0x7f672c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.004+0000 7f6749d9b640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 0x7f672c06de50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f674c142560 tx=0x7f6734009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.008+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f673c04c070 con 0x7f67380039e0 2024-04-01T00:13:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.187+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f674c142ba0 con 0x7f67380039e0 2024-04-01T00:13:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.188+0000 7f6742ffd640 1 -- 172.21.15.64:0/4232343331 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f673c07a150 con 0x7f67380039e0 2024-04-01T00:13:44.189 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:13:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 msgr2=0x7f672c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 0x7f672c06de50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f674c142560 tx=0x7f6734009040 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 msgr2=0x7f674c141720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f673c000c00 tx=0x7f673c004730 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 shutdown_connections 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f672c06b990 0x7f672c06de50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f674c13f970 0x7f674c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f674c0ca030 0x7f674c0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 --2- 172.21.15.64:0/4232343331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67380039e0 0x7f674c141720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.189+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 >> 172.21.15.64:0/4232343331 conn(0x7f674c0d0bb0 msgr2=0x7f674c143be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.190+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 shutdown_connections 2024-04-01T00:13:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.190+0000 7f6750be2640 1 -- 172.21.15.64:0/4232343331 wait complete. 2024-04-01T00:13:44.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-01T00:13:44.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.301+0000 7fe3740c0640 1 Processor -- start 2024-04-01T00:13:44.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.301+0000 7fe3740c0640 1 -- start start 2024-04-01T00:13:44.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe3740c0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe36c0d0120 0x7fe36c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe3740c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe36c15c240 0x7fe36c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe3740c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36c160ae0 con 0x7fe36c0d1960 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe3740c0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe36c0d1e90 con 0x7fe36c0d0120 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe3740c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe36c0d2170 con 0x7fe36c15c240 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe36c0d0120 0x7fe36c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe36c0d0120 0x7fe36c1605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53278/0 (socket says 172.21.15.64:53278) 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 -- 172.21.15.64:0/3188726270 learned_addr learned my addr 172.21.15.64:0/3188726270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 -- 172.21.15.64:0/3188726270 >> v2:172.21.15.100:3300/0 conn(0x7fe36c15c240 msgr2=0x7fe36c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 --2- 172.21.15.64:0/3188726270 >> v2:172.21.15.100:3300/0 conn(0x7fe36c15c240 0x7fe36c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 -- 172.21.15.64:0/3188726270 >> v1:172.21.15.100:6789/0 conn(0x7fe36c0d1960 legacy=0x7fe36c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.302+0000 7fe371e5c640 1 -- 172.21.15.64:0/3188726270 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0d2450 con 0x7fe36c0d0120 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.303+0000 7fe371e5c640 1 --2- 172.21.15.64:0/3188726270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1605a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ce4e0 tx=0x7fe36402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf785cfb509f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.303+0000 7fe370e5a640 1 -- 172.21.15.64:0/3188726270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36402e070 con 0x7fe36c0d0120 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.303+0000 7fe370e5a640 1 -- 172.21.15.64:0/3188726270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364037070 con 0x7fe36c0d0120 2024-04-01T00:13:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.303+0000 7fe370e5a640 1 -- 172.21.15.64:0/3188726270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364031a00 con 0x7fe36c0d0120 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 -- 172.21.15.64:0/3188726270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 msgr2=0x7fe36c1605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3188726270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1605a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ce4e0 tx=0x7fe36402b880 comp rx=0 tx=0).stop 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 -- 172.21.15.64:0/3188726270 shutdown_connections 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3188726270 >> v2:172.21.15.100:3300/0 conn(0x7fe36c15c240 0x7fe36c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3188726270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1605a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 -- 172.21.15.64:0/3188726270 >> 172.21.15.64:0/3188726270 conn(0x7fe36c09a980 msgr2=0x7fe36c09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 -- 172.21.15.64:0/3188726270 shutdown_connections 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.304+0000 7fe3740c0640 1 -- 172.21.15.64:0/3188726270 wait complete. 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.305+0000 7fe3740c0640 1 Processor -- start 2024-04-01T00:13:44.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.305+0000 7fe3740c0640 1 -- start start 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe36c0d1960 0x7fe36c1f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe36c15c240 0x7fe36c1f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0cc7c0 con 0x7fe36c0d0120 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0cfb40 con 0x7fe36c15c240 2024-04-01T00:13:44.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe3740c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c0cc940 con 0x7fe36c0d1960 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53288/0 (socket says 172.21.15.64:53288) 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 -- 172.21.15.64:0/3366937931 learned_addr learned my addr 172.21.15.64:0/3366937931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe37265d640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe36c15c240 0x7fe36c1f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe37165b640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe36c0d1960 0x7fe36c1f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 -- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe36c0d1960 msgr2=0x7fe36c1f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe36c0d1960 0x7fe36c1f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 -- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe36c15c240 msgr2=0x7fe36c1f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe36c15c240 0x7fe36c1f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 -- 172.21.15.64:0/3366937931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0b2930 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.306+0000 7fe371e5c640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe3640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.307+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364032880 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.307+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364032a20 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.307+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c0b2c10 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.307+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364032e10 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.307+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c06c370 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.308+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c0b3130 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.308+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe3640453f0 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.308+0000 7fe36a7fc640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 0x7fe35806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.308+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe3640ac5d0 con 0x7fe36c0d0120 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.309+0000 7fe37165b640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 0x7fe35806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.309+0000 7fe37165b640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 0x7fe35806de50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe360007fd0 tx=0x7fe360009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.313+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe364040070 con 0x7fe36c0d0120 2024-04-01T00:13:44.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.496+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe36c0681a0 con 0x7fe36c0d0120 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.496+0000 7fe36a7fc640 1 -- 172.21.15.64:0/3366937931 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fe36407aee0 con 0x7fe36c0d0120 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-04-01T00:13:44.497 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:13:44.498 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 msgr2=0x7fe35806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 0x7fe35806de50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe360007fd0 tx=0x7fe360009040 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 msgr2=0x7fe36c1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe3640046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 shutdown_connections 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe35806b990 0x7fe35806de50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe36c15c240 0x7fe36c1f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe36c0d1960 0x7fe36c1f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 --2- 172.21.15.64:0/3366937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe36c0d0120 0x7fe36c1599d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 >> 172.21.15.64:0/3366937931 conn(0x7fe36c09a980 msgr2=0x7fe36c0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:44.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 shutdown_connections 2024-04-01T00:13:44.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.498+0000 7fe3740c0640 1 -- 172.21.15.64:0/3366937931 wait complete. 2024-04-01T00:13:44.509 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-01T00:13:44.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-01T00:13:44.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.614+0000 7f0b7fae1640 1 Processor -- start 2024-04-01T00:13:44.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.614+0000 7f0b7fae1640 1 -- start start 2024-04-01T00:13:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7fae1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b780cb220 0x7f0b780cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7fae1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7815cd50 con 0x7f0b7806b4a0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7fae1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7815d030 con 0x7f0b7806b0c0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7fae1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7815fa90 con 0x7f0b780cb220 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7d07c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b7806b4a0 0x7f0b7815c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42856/0 (socket says 172.21.15.64:42856) 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7d07c640 1 -- 172.21.15.64:0/1264923000 learned_addr learned my addr 172.21.15.64:0/1264923000 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/1264923000 >> v2:172.21.15.100:3300/0 conn(0x7f0b780cb220 0x7f0b780cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237699202 0 0) 0x7f0b7815cd50 con 0x7f0b7806b4a0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.615+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b60003520 con 0x7f0b7806b4a0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1716257802 0 0) 0x7f0b60003520 con 0x7f0b7806b4a0 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 >> v2:172.21.15.100:3300/0 conn(0x7f0b780cb220 msgr2=0x7f0b780cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 --2- 172.21.15.64:0/1264923000 >> v2:172.21.15.100:3300/0 conn(0x7f0b780cb220 0x7f0b780cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 >> v1:172.21.15.100:6789/0 conn(0x7f0b7806b0c0 legacy=0x7f0b78160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7815fd70 con 0x7f0b7806b4a0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f0b6c001670 con 0x7f0b7806b4a0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 >> v1:172.21.15.64:6789/0 conn(0x7f0b7806b4a0 legacy=0x7f0b7815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b60004800 0x7f0b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b600051c0 0x7f0b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7815fa90 con 0x7f0b600039b0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6000c060 con 0x7f0b600051c0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6000c260 con 0x7f0b60004800 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d07c640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b60004800 0x7f0b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b600051c0 0x7f0b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 -- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b60004800 msgr2=0x7f0b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b60004800 0x7f0b60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 -- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b600051c0 msgr2=0x7f0b6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b600051c0 0x7f0b6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7d87d640 1 -- 172.21.15.64:0/1264923000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b60003520 con 0x7f0b600039b0 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b600051c0 0x7f0b6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.616+0000 7f0b7e07e640 1 -- 172.21.15.64:0/1264923000 reap_dead start 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.617+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b70001750 tx=0x7f0b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd195586a043284 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.617+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70001370 con 0x7f0b600039b0 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.617+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b700052d0 con 0x7f0b600039b0 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.617+0000 7f0b7c87b640 1 -- 172.21.15.64:0/1264923000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70003830 con 0x7f0b600039b0 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 -- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 msgr2=0x7f0b60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b70001750 tx=0x7f0b60005830 comp rx=0 tx=0).stop 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 -- 172.21.15.64:0/1264923000 shutdown_connections 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/1264923000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 -- 172.21.15.64:0/1264923000 >> 172.21.15.64:0/1264923000 conn(0x7f0b780d0bd0 msgr2=0x7f0b780d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 -- 172.21.15.64:0/1264923000 shutdown_connections 2024-04-01T00:13:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.618+0000 7f0b7fae1640 1 -- 172.21.15.64:0/1264923000 wait complete. 2024-04-01T00:13:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.619+0000 7f0b7fae1640 1 Processor -- start 2024-04-01T00:13:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.619+0000 7f0b7fae1640 1 -- start start 2024-04-01T00:13:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b600039b0 0x7f0b781f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b780b52b0 0x7f0b780c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780d1990 con 0x7f0b780c5e50 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780d1810 con 0x7f0b780b52b0 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0b78156570 con 0x7f0b600039b0 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53302/0 (socket says 172.21.15.64:53302) 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 -- 172.21.15.64:0/2981638257 learned_addr learned my addr 172.21.15.64:0/2981638257 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b600039b0 0x7f0b781f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7d07c640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b780b52b0 0x7f0b780c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 -- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b600039b0 msgr2=0x7f0b781f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b600039b0 0x7f0b781f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 -- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b780b52b0 msgr2=0x7f0b780c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b780b52b0 0x7f0b780c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 -- 172.21.15.64:0/2981638257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b781f21c0 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7e07e640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b78073660 tx=0x7f0b68010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68010da0 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68003990 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b781f5560 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003dc0 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.620+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b780b5a20 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.621+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b780681a0 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.622+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0b6801e020 con 0x7f0b780c5e50 2024-04-01T00:13:44.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.622+0000 7f0b767fc640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 0x7f0b6406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.622+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0b6808a930 con 0x7f0b780c5e50 2024-04-01T00:13:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.622+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 0x7f0b6406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.623+0000 7f0b7d87d640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 0x7f0b6406de50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b70033000 tx=0x7f0b70009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.627+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0b68059240 con 0x7f0b780c5e50 2024-04-01T00:13:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:44.809+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0b780b1a90 con 0x7f0b780c5e50 2024-04-01T00:13:45.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.005+0000 7f0b767fc640 1 -- 172.21.15.64:0/2981638257 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f0b6805d2e0 con 0x7f0b780c5e50 2024-04-01T00:13:45.006 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-04-01T00:13:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 msgr2=0x7f0b6406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 0x7f0b6406de50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b70033000 tx=0x7f0b70009030 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 msgr2=0x7f0b780c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b78073660 tx=0x7f0b68010590 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 shutdown_connections 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0b6406b990 0x7f0b6406de50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b780c5e50 0x7f0b780c6c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b780b52b0 0x7f0b780c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 --2- 172.21.15.64:0/2981638257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b600039b0 0x7f0b781f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.007+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 >> 172.21.15.64:0/2981638257 conn(0x7f0b780d0bd0 msgr2=0x7f0b780b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.008+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 shutdown_connections 2024-04-01T00:13:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.008+0000 7f0b7fae1640 1 -- 172.21.15.64:0/2981638257 wait complete. 2024-04-01T00:13:45.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-01T00:13:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.130+0000 7f04b6b25640 1 Processor -- start 2024-04-01T00:13:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.130+0000 7f04b6b25640 1 -- start start 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b6b25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04b00b1bb0 0x7f04b00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b6b25640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b00cdaf0 con 0x7f04b015f880 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b6b25640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b00ce310 con 0x7f04b015f4a0 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b6b25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f04b00d2aa0 con 0x7f04b00b1bb0 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04b015f880 0x7f04b00b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42858/0 (socket says 172.21.15.64:42858) 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a7fff640 1 -- 172.21.15.64:0/550272219 learned_addr learned my addr 172.21.15.64:0/550272219 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196125307 0 0) 0x7f04b00cdaf0 con 0x7f04b015f880 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0498003520 con 0x7f04b015f880 2024-04-01T00:13:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b50c2640 1 --2- 172.21.15.64:0/550272219 >> v2:172.21.15.100:3300/0 conn(0x7f04b00b1bb0 0x7f04b00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1533541553 0 0) 0x7f0498003520 con 0x7f04b015f880 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 >> v2:172.21.15.100:3300/0 conn(0x7f04b00b1bb0 msgr2=0x7f04b00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 --2- 172.21.15.64:0/550272219 >> v2:172.21.15.100:3300/0 conn(0x7f04b00b1bb0 0x7f04b00cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 >> v1:172.21.15.100:6789/0 conn(0x7f04b015f4a0 legacy=0x7f04b00cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b00ce5f0 con 0x7f04b015f880 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f04a0001670 con 0x7f04b015f880 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 >> v1:172.21.15.64:6789/0 conn(0x7f04b015f880 legacy=0x7f04b00b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04980039b0 0x7f0498003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 0x7f0498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04980051c0 0x7f049800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f049800c130 con 0x7f0498004800 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f049800c2e0 con 0x7f04980039b0 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f049800c4e0 con 0x7f04980051c0 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04a7fff640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 0x7f0498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b48c1640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04980039b0 0x7f0498003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.131+0000 7f04b50c2640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04980051c0 0x7f049800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 -- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04980051c0 msgr2=0x7f049800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04980051c0 0x7f049800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 -- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04980039b0 msgr2=0x7f0498003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04980039b0 0x7f0498003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 -- 172.21.15.64:0/550272219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498003520 con 0x7f0498004800 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04b50c2640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04980051c0 0x7f049800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04b50c2640 1 -- 172.21.15.64:0/550272219 reap_dead start 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a7fff640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 0x7f0498004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a000c210 tx=0x7f0498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd1bcc126390dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.132+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0007d10 con 0x7f0498004800 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a0030070 con 0x7f0498004800 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04a77fe640 1 -- 172.21.15.64:0/550272219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0005d70 con 0x7f0498004800 2024-04-01T00:13:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 -- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 msgr2=0x7f0498004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 0x7f0498004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a000c210 tx=0x7f0498005830 comp rx=0 tx=0).stop 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 -- 172.21.15.64:0/550272219 shutdown_connections 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 --2- 172.21.15.64:0/550272219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0498004800 0x7f0498004c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 -- 172.21.15.64:0/550272219 >> 172.21.15.64:0/550272219 conn(0x7f04b00d1d10 msgr2=0x7f04b00d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 -- 172.21.15.64:0/550272219 shutdown_connections 2024-04-01T00:13:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.133+0000 7f04b6b25640 1 -- 172.21.15.64:0/550272219 wait complete. 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.134+0000 7f04b6b25640 1 Processor -- start 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.134+0000 7f04b6b25640 1 -- start start 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0498004800 0x7f04b014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 0x7f04b0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04b00cd780 con 0x7f04b01570d0 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04b00cd600 con 0x7f04b01f41c0 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b6b25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f04b00ce0c0 con 0x7f0498004800 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b50c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b50c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53312/0 (socket says 172.21.15.64:53312) 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b50c2640 1 -- 172.21.15.64:0/1142510670 learned_addr learned my addr 172.21.15.64:0/1142510670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04a7fff640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 0x7f04b0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.135+0000 7f04b48c1640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0498004800 0x7f04b014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 -- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0498004800 msgr2=0x7f04b014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0498004800 0x7f04b014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 -- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 msgr2=0x7f04b0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 0x7f04b0159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 -- 172.21.15.64:0/1142510670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b01f4700 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04a7fff640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 0x7f04b0159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b50c2640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04b00d16f0 tx=0x7f04a8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8010e60 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8003990 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8003d60 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b01f49e0 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.136+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01f4fa0 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.137+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b01f52d0 con 0x7f04b01570d0 2024-04-01T00:13:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.137+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f04a801e020 con 0x7f04b01570d0 2024-04-01T00:13:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.138+0000 7f04a57fa640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 0x7f049c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.138+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f04a808ad70 con 0x7f04b01570d0 2024-04-01T00:13:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.138+0000 7f04b48c1640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 0x7f049c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.138+0000 7f04b48c1640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 0x7f049c06de50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000c00 tx=0x7f04ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.142+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f04a8059bf0 con 0x7f04b01570d0 2024-04-01T00:13:45.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:45.323+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f04b00688b0 con 0x7f04b01570d0 2024-04-01T00:13:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.011+0000 7f04a57fa640 1 -- 172.21.15.64:0/1142510670 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f04a80594f0 con 0x7f04b01570d0 2024-04-01T00:13:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-04-01T00:13:46.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.015+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 msgr2=0x7f049c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.015+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 0x7f049c06de50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000c00 tx=0x7f04ac009040 comp rx=0 tx=0).stop 2024-04-01T00:13:46.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.015+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 msgr2=0x7f04b0157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.015+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04b00d16f0 tx=0x7f04a8010590 comp rx=0 tx=0).stop 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 shutdown_connections 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f049c06b990 0x7f049c06de50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b01570d0 0x7f04b0157550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b01f41c0 0x7f04b0159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 --2- 172.21.15.64:0/1142510670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0498004800 0x7f04b014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.016+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 >> 172.21.15.64:0/1142510670 conn(0x7f04b00d1d10 msgr2=0x7f04b00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.017+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 shutdown_connections 2024-04-01T00:13:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.017+0000 7f04b6b25640 1 -- 172.21.15.64:0/1142510670 wait complete. 2024-04-01T00:13:46.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-01T00:13:46.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.153+0000 7f242b5c7640 1 Processor -- start 2024-04-01T00:13:46.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 -- start start 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 0x7f24240d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24240b4fb0 0x7f24240d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240d18f0 con 0x7f2424161ad0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f24240b4300 con 0x7f24240b4fb0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f242b5c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f24240b45e0 con 0x7f24241616f0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f2429b64640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2424161ad0 0x7f24240d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42872/0 (socket says 172.21.15.64:42872) 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.154+0000 7f2429b64640 1 -- 172.21.15.64:0/268732248 learned_addr learned my addr 172.21.15.64:0/268732248 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2428b62640 1 --2- 172.21.15.64:0/268732248 >> v2:172.21.15.100:3300/0 conn(0x7f24240b4fb0 0x7f24240d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2429363640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 0x7f24240d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1947320329 0 0) 0x7f24240d18f0 con 0x7f2424161ad0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f240c003520 con 0x7f2424161ad0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3679267358 0 0) 0x7f240c003520 con 0x7f2424161ad0 2024-04-01T00:13:46.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2428b62640 1 -- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 msgr2=0x7f24240d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2428b62640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 0x7f24240d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2428b62640 1 -- 172.21.15.64:0/268732248 >> v1:172.21.15.64:6789/0 conn(0x7f2424161ad0 legacy=0x7f24240d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2428b62640 1 -- 172.21.15.64:0/268732248 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240b48c0 con 0x7f24240b4fb0 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.155+0000 7f2429363640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 0x7f24240d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f2428b62640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24240b4fb0 0x7f24240d0930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a1df51a214fdb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418037070 con 0x7f24240b4fb0 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418030510 con 0x7f24240b4fb0 2024-04-01T00:13:46.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f2423fff640 1 -- 172.21.15.64:0/268732248 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140019f0 con 0x7f24240b4fb0 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f242b5c7640 1 -- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24240b4fb0 msgr2=0x7f24240d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.156+0000 7f242b5c7640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24240b4fb0 0x7f24240d0930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24180097a0 tx=0x7f241802b710 comp rx=0 tx=0).stop 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 -- 172.21.15.64:0/268732248 shutdown_connections 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24240b4fb0 0x7f24240d0930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 --2- 172.21.15.64:0/268732248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24241616f0 0x7f24240d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 -- 172.21.15.64:0/268732248 >> 172.21.15.64:0/268732248 conn(0x7f24240d3f60 msgr2=0x7f24240d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 -- 172.21.15.64:0/268732248 shutdown_connections 2024-04-01T00:13:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.157+0000 7f242b5c7640 1 -- 172.21.15.64:0/268732248 wait complete. 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 Processor -- start 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 -- start start 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24240b4fb0 0x7f24240cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24241616f0 0x7f24240c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24240d1d70 con 0x7f2424161ad0 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24240d1580 con 0x7f24241616f0 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.158+0000 7f242b5c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f24240d1ef0 con 0x7f24240b4fb0 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53322/0 (socket says 172.21.15.64:53322) 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 -- 172.21.15.64:0/24409128 learned_addr learned my addr 172.21.15.64:0/24409128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2428b62640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24241616f0 0x7f24240c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429363640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24240b4fb0 0x7f24240cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 -- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24240b4fb0 msgr2=0x7f24240cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24240b4fb0 0x7f24240cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 -- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24241616f0 msgr2=0x7f24240c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24241616f0 0x7f24240c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 -- 172.21.15.64:0/24409128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240cfb20 con 0x7f2424161ad0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2429b64640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f241400cc00 tx=0x7f241400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400ce10 con 0x7f2424161ad0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414012400 con 0x7f2424161ad0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24240cfe00 con 0x7f2424161ad0 2024-04-01T00:13:46.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140127f0 con 0x7f2424161ad0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.159+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24240d03c0 con 0x7f2424161ad0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.160+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24240681a0 con 0x7f2424161ad0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.161+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2414012990 con 0x7f2424161ad0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.161+0000 7f2421ffb640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 0x7f241006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.161+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f241408af40 con 0x7f2424161ad0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.161+0000 7f2429363640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 0x7f241006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.162+0000 7f2429363640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 0x7f241006de50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f241c009560 tx=0x7f241c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.166+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f24140594d0 con 0x7f2424161ad0 2024-04-01T00:13:46.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:13:46.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:13:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:46.386+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f24240b2a80 con 0x7f2424161ad0 2024-04-01T00:13:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.021+0000 7f2421ffb640 1 -- 172.21.15.64:0/24409128 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f241405d570 con 0x7f2424161ad0 2024-04-01T00:13:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.027+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 msgr2=0x7f241006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.027+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 0x7f241006de50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f241c009560 tx=0x7f241c009190 comp rx=0 tx=0).stop 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.027+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 msgr2=0x7f242415c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.027+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f241400cc00 tx=0x7f241400a430 comp rx=0 tx=0).stop 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 shutdown_connections 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f241006b990 0x7f241006de50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424161ad0 0x7f242415c180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24241616f0 0x7f24240c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 --2- 172.21.15.64:0/24409128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24240b4fb0 0x7f24240cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.028+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 >> 172.21.15.64:0/24409128 conn(0x7f24240d3f60 msgr2=0x7f24240b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.030+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 shutdown_connections 2024-04-01T00:13:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.030+0000 7f242b5c7640 1 -- 172.21.15.64:0/24409128 wait complete. 2024-04-01T00:13:47.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.154+0000 7f368caa4640 1 Processor -- start 2024-04-01T00:13:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.154+0000 7f368caa4640 1 -- start start 2024-04-01T00:13:47.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368caa4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36881619e0 0x7f36880d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368caa4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 0x7f36881620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368caa4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36880b3cb0 con 0x7f36880b3820 2024-04-01T00:13:47.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368caa4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f36880d2ba0 con 0x7f36881619e0 2024-04-01T00:13:47.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368caa4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f36880d2e80 con 0x7f36880cfae0 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36881619e0 0x7f36880d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f3685d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f36880b3820 0x7f36880cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42888/0 (socket says 172.21.15.64:42888) 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f3685d9b640 1 -- 172.21.15.64:0/1823267162 learned_addr learned my addr 172.21.15.64:0/1823267162 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f368659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36881619e0 0x7f36880d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53338/0 (socket says 172.21.15.64:53338) 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.155+0000 7f3686d9d640 1 --2- 172.21.15.64:0/1823267162 >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 0x7f36881620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368659c640 1 -- 172.21.15.64:0/1823267162 >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 msgr2=0x7f36881620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368659c640 1 --2- 172.21.15.64:0/1823267162 >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 0x7f36881620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368659c640 1 -- 172.21.15.64:0/1823267162 >> v1:172.21.15.64:6789/0 conn(0x7f36880b3820 legacy=0x7f36880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368659c640 1 -- 172.21.15.64:0/1823267162 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36880ceeb0 con 0x7f36881619e0 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f3686d9d640 1 --2- 172.21.15.64:0/1823267162 >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 0x7f36881620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:47.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368659c640 1 --2- 172.21.15.64:0/1823267162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36881619e0 0x7f36880d4a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f368808c1c0 tx=0x7f368002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dbbcb36d15b5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368559a640 1 -- 172.21.15.64:0/1823267162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36800315a0 con 0x7f36881619e0 2024-04-01T00:13:47.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368559a640 1 -- 172.21.15.64:0/1823267162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3680037070 con 0x7f36881619e0 2024-04-01T00:13:47.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.156+0000 7f368559a640 1 -- 172.21.15.64:0/1823267162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680031960 con 0x7f36881619e0 2024-04-01T00:13:47.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 -- 172.21.15.64:0/1823267162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36881619e0 msgr2=0x7f36880d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 --2- 172.21.15.64:0/1823267162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36881619e0 0x7f36880d4a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f368808c1c0 tx=0x7f368002b840 comp rx=0 tx=0).stop 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 -- 172.21.15.64:0/1823267162 shutdown_connections 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 --2- 172.21.15.64:0/1823267162 >> v2:172.21.15.100:3300/0 conn(0x7f36880cfae0 0x7f36881620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 --2- 172.21.15.64:0/1823267162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36881619e0 0x7f36880d4a60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 -- 172.21.15.64:0/1823267162 >> 172.21.15.64:0/1823267162 conn(0x7f368809d290 msgr2=0x7f36880d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 -- 172.21.15.64:0/1823267162 shutdown_connections 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.157+0000 7f368caa4640 1 -- 172.21.15.64:0/1823267162 wait complete. 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.158+0000 7f368caa4640 1 Processor -- start 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.158+0000 7f368caa4640 1 -- start start 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36880cfae0 0x7f36880b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36881619e0 0x7f36880b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36880b31b0 con 0x7f36880b3820 2024-04-01T00:13:47.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36880d2890 con 0x7f36881619e0 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368caa4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f36880ceb10 con 0x7f36880cfae0 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53346/0 (socket says 172.21.15.64:53346) 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 -- 172.21.15.64:0/2367872142 learned_addr learned my addr 172.21.15.64:0/2367872142 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f3686d9d640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36881619e0 0x7f36880b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 -- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36880cfae0 msgr2=0x7f36880b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36880cfae0 0x7f36880b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 -- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36881619e0 msgr2=0x7f36880b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36881619e0 0x7f36880b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.159+0000 7f368659c640 1 -- 172.21.15.64:0/2367872142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368814f940 con 0x7f36880b3820 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f368659c640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f36800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680032840 con 0x7f36880b3820 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3680032a00 con 0x7f36880b3820 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36881f7ae0 con 0x7f36880b3820 2024-04-01T00:13:47.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680032df0 con 0x7f36880b3820 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.160+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36881f80a0 con 0x7f36880b3820 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.161+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36880681a0 con 0x7f36880b3820 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.161+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f368004d3f0 con 0x7f36880b3820 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.162+0000 7f367effd640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 0x7f366806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.162+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f368002f030 con 0x7f36880b3820 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.162+0000 7f3685d9b640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 0x7f366806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.163+0000 7f3685d9b640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 0x7f366806de50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f60 tx=0x7f3670007a80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.166+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f368814f940 con 0x7f36880b3820 2024-04-01T00:13:47.321 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T00:13:47.319+0000 7fe784f9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-01T00:13:47.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.348+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36880d2100 con 0x7f36880b3820 2024-04-01T00:13:47.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.349+0000 7f367effd640 1 -- 172.21.15.64:0/2367872142 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16594 (secure 0 0 0) 0x7f36800a90f0 con 0x7f36880b3820 2024-04-01T00:13:47.350 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:47.350 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:46.387394+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 msgr2=0x7f366806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 0x7f366806de50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f60 tx=0x7f3670007a80 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 msgr2=0x7f368814f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f36800046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 shutdown_connections 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f366806b990 0x7f366806de50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36881619e0 0x7f36880b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36880cfae0 0x7f36880b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 --2- 172.21.15.64:0/2367872142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36880b3820 0x7f368814f400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 >> 172.21.15.64:0/2367872142 conn(0x7f368809d290 msgr2=0x7f3688066e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 shutdown_connections 2024-04-01T00:13:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.351+0000 7f368caa4640 1 -- 172.21.15.64:0/2367872142 wait complete. 2024-04-01T00:13:47.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.464+0000 7f4815565640 1 Processor -- start 2024-04-01T00:13:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.464+0000 7f4815565640 1 -- start start 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4815565640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48100723d0 0x7f48100b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4815565640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 0x7f48100cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4815565640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481015fde0 con 0x7f48100d0480 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4815565640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f48101600c0 con 0x7f48100723d0 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4815565640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f48101603a0 con 0x7f4810070eb0 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f48100d0480 0x7f48100cc620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42890/0 (socket says 172.21.15.64:42890) 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480f7fe640 1 -- 172.21.15.64:0/853029715 learned_addr learned my addr 172.21.15.64:0/853029715 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480effd640 1 --2- 172.21.15.64:0/853029715 >> v2:172.21.15.100:3300/0 conn(0x7f48100723d0 0x7f48100b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4807fff640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 0x7f48100cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480e7fc640 1 -- 172.21.15.64:0/853029715 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022344975 0 0) 0x7f481015fde0 con 0x7f48100d0480 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480e7fc640 1 -- 172.21.15.64:0/853029715 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc003520 con 0x7f48100d0480 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480effd640 1 -- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 msgr2=0x7f48100cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480effd640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 0x7f48100cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480effd640 1 -- 172.21.15.64:0/853029715 >> v1:172.21.15.64:6789/0 conn(0x7f48100d0480 legacy=0x7f48100cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f480effd640 1 -- 172.21.15.64:0/853029715 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4810160680 con 0x7f48100723d0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.465+0000 7f4807fff640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 0x7f48100cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.466+0000 7f480effd640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100723d0 0x7f48100b3820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48000097a0 tx=0x7f480002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ee8210e180f933 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.466+0000 7f480e7fc640 1 -- 172.21.15.64:0/853029715 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000315a0 con 0x7f48100723d0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.466+0000 7f480e7fc640 1 -- 172.21.15.64:0/853029715 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800037070 con 0x7f48100723d0 2024-04-01T00:13:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.466+0000 7f480e7fc640 1 -- 172.21.15.64:0/853029715 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800031960 con 0x7f48100723d0 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 -- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100723d0 msgr2=0x7f48100b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100723d0 0x7f48100b3820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48000097a0 tx=0x7f480002b840 comp rx=0 tx=0).stop 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 -- 172.21.15.64:0/853029715 shutdown_connections 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4810070eb0 0x7f48100cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 --2- 172.21.15.64:0/853029715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100723d0 0x7f48100b3820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 -- 172.21.15.64:0/853029715 >> 172.21.15.64:0/853029715 conn(0x7f481006b4e0 msgr2=0x7f481006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 -- 172.21.15.64:0/853029715 shutdown_connections 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.467+0000 7f4815565640 1 -- 172.21.15.64:0/853029715 wait complete. 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.468+0000 7f4815565640 1 Processor -- start 2024-04-01T00:13:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.468+0000 7f4815565640 1 -- start start 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48100723d0 0x7f48101e5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100d0480 0x7f4810156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48100718a0 con 0x7f4810070eb0 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4810071720 con 0x7f48100d0480 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f48100727b0 con 0x7f48100723d0 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53350/0 (socket says 172.21.15.64:53350) 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 -- 172.21.15.64:0/238113930 learned_addr learned my addr 172.21.15.64:0/238113930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480f7fe640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100d0480 0x7f4810156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4807fff640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48100723d0 0x7f48101e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 -- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48100723d0 msgr2=0x7f48101e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48100723d0 0x7f48101e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 -- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100d0480 msgr2=0x7f4810156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100d0480 0x7f4810156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 -- 172.21.15.64:0/238113930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48101e5ec0 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f480effd640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4800034040 tx=0x7f48000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800032630 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48000327f0 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.469+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48101e61a0 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.470+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800032be0 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.470+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101e6760 con 0x7f4810070eb0 2024-04-01T00:13:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.470+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48100681a0 con 0x7f4810070eb0 2024-04-01T00:13:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.471+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f480004e430 con 0x7f4810070eb0 2024-04-01T00:13:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.471+0000 7f48077fe640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 0x7f47f006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.471+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f480002f030 con 0x7f4810070eb0 2024-04-01T00:13:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.471+0000 7f4807fff640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 0x7f47f006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.472+0000 7f4807fff640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 0x7f47f006de50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009560 tx=0x7f47f8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.476+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4800040070 con 0x7f4810070eb0 2024-04-01T00:13:47.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.658+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48100ce250 con 0x7f4810070eb0 2024-04-01T00:13:47.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.659+0000 7f48077fe640 1 -- 172.21.15.64:0/238113930 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16594 (secure 0 0 0) 0x7f480007aaa0 con 0x7f4810070eb0 2024-04-01T00:13:47.660 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:47.660 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:46.387394+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:47.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 msgr2=0x7f47f006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 0x7f47f006de50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009560 tx=0x7f47f8009190 comp rx=0 tx=0).stop 2024-04-01T00:13:47.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 msgr2=0x7f4810150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4800034040 tx=0x7f48000046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 shutdown_connections 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f47f006b990 0x7f47f006de50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48100d0480 0x7f4810156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48100723d0 0x7f48101e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 --2- 172.21.15.64:0/238113930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4810070eb0 0x7f4810150740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 >> 172.21.15.64:0/238113930 conn(0x7f481006b4e0 msgr2=0x7f481015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 shutdown_connections 2024-04-01T00:13:47.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.661+0000 7f4815565640 1 -- 172.21.15.64:0/238113930 wait complete. 2024-04-01T00:13:47.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-01T00:13:47.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.777+0000 7fbff6363640 1 Processor -- start 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.777+0000 7fbff6363640 1 -- start start 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff6363640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbff00d4a40 0x7fbff00b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff6363640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbff00d0900 0x7fbff00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff6363640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff0161e10 con 0x7fbff0161a30 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff6363640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fbff00cef20 con 0x7fbff00d0900 2024-04-01T00:13:47.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff6363640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbff00d2880 con 0x7fbff00d4a40 2024-04-01T00:13:47.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbff00d0900 0x7fbff00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbff00d0900 0x7fbff00d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53356/0 (socket says 172.21.15.64:53356) 2024-04-01T00:13:47.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbfeffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbff0161a30 0x7fbff00b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42902/0 (socket says 172.21.15.64:42902) 2024-04-01T00:13:47.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 -- 172.21.15.64:0/1791795972 learned_addr learned my addr 172.21.15.64:0/1791795972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:47.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbfef7fe640 1 --2- 172.21.15.64:0/1791795972 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d4a40 0x7fbff00b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 -- 172.21.15.64:0/1791795972 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d4a40 msgr2=0x7fbff00b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 --2- 172.21.15.64:0/1791795972 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d4a40 0x7fbff00b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 -- 172.21.15.64:0/1791795972 >> v1:172.21.15.64:6789/0 conn(0x7fbff0161a30 legacy=0x7fbff00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbff4900640 1 -- 172.21.15.64:0/1791795972 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff00d2b20 con 0x7fbff00d0900 2024-04-01T00:13:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.778+0000 7fbfeeffd640 1 -- 172.21.15.64:0/1791795972 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2189940979 0 0) 0x7fbfe0001110 con 0x7fbff0161a30 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff4900640 1 --2- 172.21.15.64:0/1791795972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d0900 0x7fbff00d0d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009e50 tx=0x7fbfe802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2f2182e3c876ad server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbfeeffd640 1 -- 172.21.15.64:0/1791795972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe802cb20 con 0x7fbff00d0900 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbfeeffd640 1 -- 172.21.15.64:0/1791795972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe802ccc0 con 0x7fbff00d0900 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbfeeffd640 1 -- 172.21.15.64:0/1791795972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8034610 con 0x7fbff00d0900 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 -- 172.21.15.64:0/1791795972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d0900 msgr2=0x7fbff00d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 --2- 172.21.15.64:0/1791795972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d0900 0x7fbff00d0d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009e50 tx=0x7fbfe802bf80 comp rx=0 tx=0).stop 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 -- 172.21.15.64:0/1791795972 shutdown_connections 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 --2- 172.21.15.64:0/1791795972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d0900 0x7fbff00d0d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 --2- 172.21.15.64:0/1791795972 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d4a40 0x7fbff00b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 -- 172.21.15.64:0/1791795972 >> 172.21.15.64:0/1791795972 conn(0x7fbff009d250 msgr2=0x7fbff00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 -- 172.21.15.64:0/1791795972 shutdown_connections 2024-04-01T00:13:47.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.779+0000 7fbff6363640 1 -- 172.21.15.64:0/1791795972 wait complete. 2024-04-01T00:13:47.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 Processor -- start 2024-04-01T00:13:47.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 -- start start 2024-04-01T00:13:47.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0900 0x7fbff015b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0161a30 0x7fbff01f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbff00cf3b0 con 0x7fbff00d4a40 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbff00ceb40 con 0x7fbff00d0900 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.781+0000 7fbff6363640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbff00cf530 con 0x7fbff0161a30 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53358/0 (socket says 172.21.15.64:53358) 2024-04-01T00:13:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 -- 172.21.15.64:0/996446954 learned_addr learned my addr 172.21.15.64:0/996446954 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:47.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfeffff640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0900 0x7fbff015b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbff4900640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0161a30 0x7fbff01f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 -- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0161a30 msgr2=0x7fbff01f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0161a30 0x7fbff01f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 -- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0900 msgr2=0x7fbff015b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0900 0x7fbff015b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 -- 172.21.15.64:0/996446954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff015ba30 con 0x7fbff00d4a40 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfef7fe640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9b0 tx=0x7fbfe400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4017070 con 0x7fbff00d4a40 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe400c9a0 con 0x7fbff00d4a40 2024-04-01T00:13:47.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff00cdf30 con 0x7fbff00d4a40 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400cd90 con 0x7fbff00d4a40 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.782+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff00cb650 con 0x7fbff00d4a40 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.783+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff00681a0 con 0x7fbff00d4a40 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.784+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fbfe401a070 con 0x7fbff00d4a40 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.784+0000 7fbfecff9640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 0x7fbfdc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:47.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.784+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbfe408a740 con 0x7fbff00d4a40 2024-04-01T00:13:47.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.784+0000 7fbfeffff640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 0x7fbfdc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:47.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.784+0000 7fbfeffff640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 0x7fbfdc06de80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007fd0 tx=0x7fbfe0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:47.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:47.788+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fbff015ba30 con 0x7fbff00d4a40 2024-04-01T00:13:48.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.008+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbff00b4320 con 0x7fbff00d4a40 2024-04-01T00:13:48.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.318+0000 7fbfecff9640 1 -- 172.21.15.64:0/996446954 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fbfe4058d90 con 0x7fbff00d4a40 2024-04-01T00:13:48.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 msgr2=0x7fbfdc06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 0x7fbfdc06de80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007fd0 tx=0x7fbfe0007de0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 msgr2=0x7fbff00cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9b0 tx=0x7fbfe400ee70 comp rx=0 tx=0).stop 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 shutdown_connections 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbfdc06b9c0 0x7fbfdc06de80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0161a30 0x7fbff01f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff00d4a40 0x7fbff00cb9e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 --2- 172.21.15.64:0/996446954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0900 0x7fbff015b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 >> 172.21.15.64:0/996446954 conn(0x7fbff009d250 msgr2=0x7fbff006cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.320+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 shutdown_connections 2024-04-01T00:13:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.321+0000 7fbff6363640 1 -- 172.21.15.64:0/996446954 wait complete. 2024-04-01T00:13:48.334 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-01T00:13:48.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-01T00:13:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.440+0000 7fd7a449d640 1 Processor -- start 2024-04-01T00:13:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.440+0000 7fd7a449d640 1 -- start start 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a449d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd79c0d1010 0x7fd79c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a449d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c161e10 con 0x7fd79c161a30 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a449d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c0d2880 con 0x7fd79c0d4a40 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a449d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c0cef20 con 0x7fd79c0d1010 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a2a3a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd79c0d1010 0x7fd79c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a2a3a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd79c0d1010 0x7fd79c0b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53372/0 (socket says 172.21.15.64:53372) 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a2239640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd79c161a30 0x7fd79c0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42912/0 (socket says 172.21.15.64:42912) 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.441+0000 7fd7a2a3a640 1 -- 172.21.15.64:0/990210789 learned_addr learned my addr 172.21.15.64:0/990210789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a2a3a640 1 -- 172.21.15.64:0/990210789 >> v1:172.21.15.100:6789/0 conn(0x7fd79c0d4a40 legacy=0x7fd79c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a2a3a640 1 -- 172.21.15.64:0/990210789 >> v1:172.21.15.64:6789/0 conn(0x7fd79c161a30 legacy=0x7fd79c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a1237640 1 -- 172.21.15.64:0/990210789 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274124059 0 0) 0x7fd79c0d2880 con 0x7fd79c0d4a40 2024-04-01T00:13:48.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a2a3a640 1 -- 172.21.15.64:0/990210789 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c0d2b20 con 0x7fd79c0d1010 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a2a3a640 1 --2- 172.21.15.64:0/990210789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d1010 0x7fd79c0b3740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009e50 tx=0x7fd78c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51589879d9a3db4e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a1237640 1 -- 172.21.15.64:0/990210789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c02c960 con 0x7fd79c0d1010 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a1237640 1 -- 172.21.15.64:0/990210789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c02cb00 con 0x7fd79c0d1010 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.442+0000 7fd7a1237640 1 -- 172.21.15.64:0/990210789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0343f0 con 0x7fd79c0d1010 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 -- 172.21.15.64:0/990210789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d1010 msgr2=0x7fd79c0b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 --2- 172.21.15.64:0/990210789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d1010 0x7fd79c0b3740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009e50 tx=0x7fd78c02be40 comp rx=0 tx=0).stop 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 -- 172.21.15.64:0/990210789 shutdown_connections 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 --2- 172.21.15.64:0/990210789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d1010 0x7fd79c0b3740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 -- 172.21.15.64:0/990210789 >> 172.21.15.64:0/990210789 conn(0x7fd79c09d250 msgr2=0x7fd79c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:48.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 -- 172.21.15.64:0/990210789 shutdown_connections 2024-04-01T00:13:48.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.443+0000 7fd7a449d640 1 -- 172.21.15.64:0/990210789 wait complete. 2024-04-01T00:13:48.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.444+0000 7fd7a449d640 1 Processor -- start 2024-04-01T00:13:48.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.444+0000 7fd7a449d640 1 -- start start 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd79c0d1010 0x7fd79c0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c161a30 0x7fd79c160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0cf3b0 con 0x7fd79c0d4a40 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0ceb40 con 0x7fd79c161a30 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a449d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c0cf530 con 0x7fd79c0d1010 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53376/0 (socket says 172.21.15.64:53376) 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 -- 172.21.15.64:0/4239359953 learned_addr learned my addr 172.21.15.64:0/4239359953 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a2a3a640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c161a30 0x7fd79c160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a2239640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd79c0d1010 0x7fd79c0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 -- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd79c0d1010 msgr2=0x7fd79c0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd79c0d1010 0x7fd79c0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 -- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c161a30 msgr2=0x7fd79c160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c161a30 0x7fd79c160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 -- 172.21.15.64:0/4239359953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1f7830 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.445+0000 7fd7a1a38640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c4b0 tx=0x7fd79000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.446+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790017070 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.446+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790005480 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.446+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1f7b10 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.446+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790003830 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.446+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1f80d0 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.447+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c0c93d0 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.447+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd79001a070 con 0x7fd79c0d4a40 2024-04-01T00:13:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.447+0000 7fd79affd640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 0x7fd78806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:48.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.448+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd790089ae0 con 0x7fd79c0d4a40 2024-04-01T00:13:48.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.448+0000 7fd7a2239640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 0x7fd78806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:48.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.448+0000 7fd7a2239640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 0x7fd78806de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd794000f60 tx=0x7fd794007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:48.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.452+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd79c0c93d0 con 0x7fd79c0d4a40 2024-04-01T00:13:48.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:48.670+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd79c0681a0 con 0x7fd79c0d4a40 2024-04-01T00:13:49.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.325+0000 7fd79affd640 1 -- 172.21.15.64:0/4239359953 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fd790058130 con 0x7fd79c0d4a40 2024-04-01T00:13:49.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 msgr2=0x7fd78806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 0x7fd78806de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd794000f60 tx=0x7fd794007e20 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 msgr2=0x7fd79c0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c4b0 tx=0x7fd79000c970 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 shutdown_connections 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd78806b990 0x7fd78806de50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c161a30 0x7fd79c160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd79c0d4a40 0x7fd79c0cd730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 --2- 172.21.15.64:0/4239359953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd79c0d1010 0x7fd79c0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 >> 172.21.15.64:0/4239359953 conn(0x7fd79c09d250 msgr2=0x7fd79c15bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 shutdown_connections 2024-04-01T00:13:49.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.328+0000 7fd7a449d640 1 -- 172.21.15.64:0/4239359953 wait complete. 2024-04-01T00:13:49.341 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-01T00:13:49.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:13:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.448+0000 7fc41dcaf640 1 Processor -- start 2024-04-01T00:13:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.448+0000 7fc41dcaf640 1 -- start start 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc41dcaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc41dcaf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4180cff40 0x7fc4180d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc41dcaf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc418071460 con 0x7fc41815f610 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc41dcaf640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fc41815f180 con 0x7fc4180cff40 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc41dcaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc418071740 con 0x7fc418151fe0 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4180cff40 0x7fc4180d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc416ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc41815f610 0x7fc418070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42916/0 (socket says 172.21.15.64:42916) 2024-04-01T00:13:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4180cff40 0x7fc4180d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53386/0 (socket says 172.21.15.64:53386) 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc416ffd640 1 -- 172.21.15.64:0/3392089203 learned_addr learned my addr 172.21.15.64:0/3392089203 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc4177fe640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 -- 172.21.15.64:0/3392089203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 msgr2=0x7fc41815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41815beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 -- 172.21.15.64:0/3392089203 >> v1:172.21.15.64:6789/0 conn(0x7fc41815f610 legacy=0x7fc418070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.449+0000 7fc417fff640 1 -- 172.21.15.64:0/3392089203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41815eb30 con 0x7fc4180cff40 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.450+0000 7fc417fff640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4180cff40 0x7fc4180d03d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc410009d90 tx=0x7fc41002e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c3b63728d3218c server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.450+0000 7fc4167fc640 1 -- 172.21.15.64:0/3392089203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41003b070 con 0x7fc4180cff40 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.450+0000 7fc4167fc640 1 -- 172.21.15.64:0/3392089203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc410036440 con 0x7fc4180cff40 2024-04-01T00:13:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.450+0000 7fc4167fc640 1 -- 172.21.15.64:0/3392089203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41002ee60 con 0x7fc4180cff40 2024-04-01T00:13:49.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 -- 172.21.15.64:0/3392089203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4180cff40 msgr2=0x7fc4180d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4180cff40 0x7fc4180d03d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc410009d90 tx=0x7fc41002e0b0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 -- 172.21.15.64:0/3392089203 shutdown_connections 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4180cff40 0x7fc4180d03d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/3392089203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 -- 172.21.15.64:0/3392089203 >> 172.21.15.64:0/3392089203 conn(0x7fc41806b4e0 msgr2=0x7fc41806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 -- 172.21.15.64:0/3392089203 shutdown_connections 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.451+0000 7fc41dcaf640 1 -- 172.21.15.64:0/3392089203 wait complete. 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.452+0000 7fc41dcaf640 1 Processor -- start 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.452+0000 7fc41dcaf640 1 -- start start 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180cff40 0x7fc41814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41809d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180cc740 con 0x7fc41815f610 2024-04-01T00:13:49.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180cc5c0 con 0x7fc4180cff40 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc41dcaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc4180cc8c0 con 0x7fc418151fe0 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53390/0 (socket says 172.21.15.64:53390) 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 -- 172.21.15.64:0/1794115067 learned_addr learned my addr 172.21.15.64:0/1794115067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc416ffd640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41809d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc4177fe640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180cff40 0x7fc41814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 -- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 msgr2=0x7fc41809d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41809d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 -- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180cff40 msgr2=0x7fc41814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.453+0000 7fc417fff640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180cff40 0x7fc41814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc417fff640 1 -- 172.21.15.64:0/1794115067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41809ad60 con 0x7fc41815f610 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc417fff640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc410009d90 tx=0x7fc410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41003b050 con 0x7fc41815f610 2024-04-01T00:13:49.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc410047070 con 0x7fc41815f610 2024-04-01T00:13:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410040a70 con 0x7fc41815f610 2024-04-01T00:13:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41809b040 con 0x7fc41815f610 2024-04-01T00:13:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.454+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41809b600 con 0x7fc41815f610 2024-04-01T00:13:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.455+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41814ed20 con 0x7fc41815f610 2024-04-01T00:13:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.455+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fc410040c10 con 0x7fc41815f610 2024-04-01T00:13:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.456+0000 7fc3fffff640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 0x7fc40406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.456+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc4100af6c0 con 0x7fc41815f610 2024-04-01T00:13:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.456+0000 7fc4177fe640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 0x7fc40406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.456+0000 7fc4177fe640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 0x7fc40406de50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc408009b40 tx=0x7fc408009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.460+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fc41814ed20 con 0x7fc41815f610 2024-04-01T00:13:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.682+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4180681a0 con 0x7fc41815f610 2024-04-01T00:13:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.683+0000 7fc3fffff640 1 -- 172.21.15.64:0/1794115067 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fc41007dd10 con 0x7fc41815f610 2024-04-01T00:13:49.684 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:49.684 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:13:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:13:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 msgr2=0x7fc40406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 0x7fc40406de50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc408009b40 tx=0x7fc408009190 comp rx=0 tx=0).stop 2024-04-01T00:13:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 msgr2=0x7fc4181e5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc410009d90 tx=0x7fc410004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 shutdown_connections 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc40406b990 0x7fc40406de50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc41815f610 0x7fc4181e5a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418151fe0 0x7fc41809d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 --2- 172.21.15.64:0/1794115067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180cff40 0x7fc41814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 >> 172.21.15.64:0/1794115067 conn(0x7fc41806b4e0 msgr2=0x7fc41806b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 shutdown_connections 2024-04-01T00:13:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.685+0000 7fc41dcaf640 1 -- 172.21.15.64:0/1794115067 wait complete. 2024-04-01T00:13:49.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.798+0000 7f6110015640 1 Processor -- start 2024-04-01T00:13:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f6110015640 1 -- start start 2024-04-01T00:13:49.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f6110015640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f61080d4a00 0x7f61080ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f6110015640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61080d1ff0 con 0x7f61080b6dc0 2024-04-01T00:13:49.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f6110015640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61080cf280 con 0x7f61080d08f0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f6110015640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f61080b5830 con 0x7f61080d4a00 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f610e5b2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61080b6dc0 0x7f61080b7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42922/0 (socket says 172.21.15.64:42922) 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f610e5b2640 1 -- 172.21.15.64:0/2983047677 learned_addr learned my addr 172.21.15.64:0/2983047677 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.799+0000 7f610d5b0640 1 --2- 172.21.15.64:0/2983047677 >> v2:172.21.15.100:3300/0 conn(0x7f61080d4a00 0x7f61080ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300035047 0 0) 0x7f6100001490 con 0x7f61080b6dc0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f4003520 con 0x7f61080b6dc0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004065139 0 0) 0x7f61080cf280 con 0x7f61080d08f0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6100001490 con 0x7f61080d08f0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610d5b0640 1 -- 172.21.15.64:0/2983047677 >> v1:172.21.15.100:6789/0 conn(0x7f61080d08f0 legacy=0x7f61080d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610d5b0640 1 -- 172.21.15.64:0/2983047677 >> v1:172.21.15.64:6789/0 conn(0x7f61080b6dc0 legacy=0x7f61080b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.800+0000 7f610d5b0640 1 -- 172.21.15.64:0/2983047677 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61080cf560 con 0x7f61080d4a00 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f610d5b0640 1 --2- 172.21.15.64:0/2983047677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080d4a00 0x7f61080ceba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60f80097a0 tx=0x7f60f802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed15cad0af1f3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8034070 con 0x7f61080d4a00 2024-04-01T00:13:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f8030610 con 0x7f61080d4a00 2024-04-01T00:13:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f610cdaf640 1 -- 172.21.15.64:0/2983047677 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80309e0 con 0x7f61080d4a00 2024-04-01T00:13:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 -- 172.21.15.64:0/2983047677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080d4a00 msgr2=0x7f61080ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 --2- 172.21.15.64:0/2983047677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080d4a00 0x7f61080ceba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60f80097a0 tx=0x7f60f802b960 comp rx=0 tx=0).stop 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 -- 172.21.15.64:0/2983047677 shutdown_connections 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 --2- 172.21.15.64:0/2983047677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080d4a00 0x7f61080ceba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 -- 172.21.15.64:0/2983047677 >> 172.21.15.64:0/2983047677 conn(0x7f61080d3ee0 msgr2=0x7f61080d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 -- 172.21.15.64:0/2983047677 shutdown_connections 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.801+0000 7f6110015640 1 -- 172.21.15.64:0/2983047677 wait complete. 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 Processor -- start 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 -- start start 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080b6dc0 0x7f610809dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61080d4a00 0x7f610815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61080cfb20 con 0x7f61080d08f0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61080d4e60 con 0x7f61080b6dc0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f6110015640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f61080cfe00 con 0x7f61080d4a00 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f610d5b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f610d5b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53396/0 (socket says 172.21.15.64:53396) 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f610d5b0640 1 -- 172.21.15.64:0/3846297556 learned_addr learned my addr 172.21.15.64:0/3846297556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f610ddb1640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080b6dc0 0x7f610809dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.803+0000 7f610e5b2640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61080d4a00 0x7f610815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 -- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61080d4a00 msgr2=0x7f610815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61080d4a00 0x7f610815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 -- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080b6dc0 msgr2=0x7f610809dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080b6dc0 0x7f610809dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 -- 172.21.15.64:0/3846297556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61080cd8f0 con 0x7f61080d08f0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f610d5b0640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60f8000c00 tx=0x7f60f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f804a070 con 0x7f61080d08f0 2024-04-01T00:13:49.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f8030410 con 0x7f61080d08f0 2024-04-01T00:13:49.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61080cdbd0 con 0x7f61080d08f0 2024-04-01T00:13:49.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8034040 con 0x7f61080d08f0 2024-04-01T00:13:49.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.804+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61080a1770 con 0x7f61080d08f0 2024-04-01T00:13:49.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.805+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61080689a0 con 0x7f61080d08f0 2024-04-01T00:13:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.805+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f60f804d070 con 0x7f61080d08f0 2024-04-01T00:13:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.806+0000 7f60fe7fc640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 0x7f60ec06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.806+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f60f80ac330 con 0x7f61080d08f0 2024-04-01T00:13:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.806+0000 7f610ddb1640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 0x7f60ec06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.806+0000 7f610ddb1640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 0x7f60ec06de50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6104007fd0 tx=0x7f6104009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:49.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.810+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f61080689a0 con 0x7f61080d08f0 2024-04-01T00:13:49.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.992+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61080681a0 con 0x7f61080d08f0 2024-04-01T00:13:49.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.993+0000 7f60fe7fc640 1 -- 172.21.15.64:0/3846297556 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f60f807a8f0 con 0x7f61080d08f0 2024-04-01T00:13:49.994 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:49.994 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:49.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 msgr2=0x7f60ec06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 0x7f60ec06de50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6104007fd0 tx=0x7f6104009040 comp rx=0 tx=0).stop 2024-04-01T00:13:49.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 msgr2=0x7f61080a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60f8000c00 tx=0x7f60f80046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 shutdown_connections 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f60ec06b990 0x7f60ec06de50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61080d4a00 0x7f610815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61080d08f0 0x7f61080a1230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 --2- 172.21.15.64:0/3846297556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61080b6dc0 0x7f610809dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 >> 172.21.15.64:0/3846297556 conn(0x7f61080d3ee0 msgr2=0x7f610806bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 shutdown_connections 2024-04-01T00:13:49.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:49.995+0000 7f6110015640 1 -- 172.21.15.64:0/3846297556 wait complete. 2024-04-01T00:13:50.007 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-01T00:13:50.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.108+0000 7fd6f14ae640 1 Processor -- start 2024-04-01T00:13:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.108+0000 7fd6f14ae640 1 -- start start 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6f14ae640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd6ec15f610 0x7fd6ec15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6f14ae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 0x7fd6ec0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6f14ae640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ec071460 con 0x7fd6ec151fe0 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6f14ae640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ec0d08e0 con 0x7fd6ec15f610 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6f14ae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec15f180 con 0x7fd6ec0cff40 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd6ec151fe0 0x7fd6ec070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42930/0 (socket says 172.21.15.64:42930) 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eaffd640 1 -- 172.21.15.64:0/3869342703 learned_addr learned my addr 172.21.15.64:0/3869342703 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6e3fff640 1 --2- 172.21.15.64:0/3869342703 >> v2:172.21.15.100:3300/0 conn(0x7fd6ec15f610 0x7fd6ec15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/3869342703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 0x7fd6ec0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6ea7fc640 1 -- 172.21.15.64:0/3869342703 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838432196 0 0) 0x7fd6ec071460 con 0x7fd6ec151fe0 2024-04-01T00:13:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6ea7fc640 1 -- 172.21.15.64:0/3869342703 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6d8003520 con 0x7fd6ec151fe0 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/3869342703 >> v2:172.21.15.100:3300/0 conn(0x7fd6ec15f610 msgr2=0x7fd6ec15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/3869342703 >> v2:172.21.15.100:3300/0 conn(0x7fd6ec15f610 0x7fd6ec15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/3869342703 >> v1:172.21.15.64:6789/0 conn(0x7fd6ec151fe0 legacy=0x7fd6ec070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.109+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/3869342703 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec071740 con 0x7fd6ec0cff40 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.110+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/3869342703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 0x7fd6ec0d03a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009d90 tx=0x7fd6e402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f768d64c2c72c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.110+0000 7fd6ea7fc640 1 -- 172.21.15.64:0/3869342703 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4039070 con 0x7fd6ec0cff40 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.110+0000 7fd6ea7fc640 1 -- 172.21.15.64:0/3869342703 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e402cc50 con 0x7fd6ec0cff40 2024-04-01T00:13:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.110+0000 7fd6ea7fc640 1 -- 172.21.15.64:0/3869342703 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034440 con 0x7fd6ec0cff40 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 -- 172.21.15.64:0/3869342703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 msgr2=0x7fd6ec0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/3869342703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 0x7fd6ec0d03a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009d90 tx=0x7fd6e402bea0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 -- 172.21.15.64:0/3869342703 shutdown_connections 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/3869342703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec0cff40 0x7fd6ec0d03a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/3869342703 >> v2:172.21.15.100:3300/0 conn(0x7fd6ec15f610 0x7fd6ec15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 -- 172.21.15.64:0/3869342703 >> 172.21.15.64:0/3869342703 conn(0x7fd6ec06b4e0 msgr2=0x7fd6ec06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 -- 172.21.15.64:0/3869342703 shutdown_connections 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.111+0000 7fd6f14ae640 1 -- 172.21.15.64:0/3869342703 wait complete. 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.112+0000 7fd6f14ae640 1 Processor -- start 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.112+0000 7fd6f14ae640 1 -- start start 2024-04-01T00:13:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.112+0000 7fd6f14ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0cff40 0x7fd6ec150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec151fe0 0x7fd6ec151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec164af0 con 0x7fd6ec15f610 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec15ee10 con 0x7fd6ec0cff40 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec164c70 con 0x7fd6ec151fe0 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53398/0 (socket says 172.21.15.64:53398) 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/604821922 learned_addr learned my addr 172.21.15.64:0/604821922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eaffd640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0cff40 0x7fd6ec150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6e3fff640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec151fe0 0x7fd6ec151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec151fe0 msgr2=0x7fd6ec151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec151fe0 0x7fd6ec151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0cff40 msgr2=0x7fd6ec150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0cff40 0x7fd6ec150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 -- 172.21.15.64:0/604821922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0b4b00 con 0x7fd6ec15f610 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6eb7fe640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009d90 tx=0x7fd6e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4039050 con 0x7fd6ec15f610 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e4045070 con 0x7fd6ec15f610 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec0b4de0 con 0x7fd6ec15f610 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4009450 con 0x7fd6ec15f610 2024-04-01T00:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.113+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec152900 con 0x7fd6ec15f610 2024-04-01T00:13:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.114+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec152410 con 0x7fd6ec15f610 2024-04-01T00:13:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.115+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd6e403e610 con 0x7fd6ec15f610 2024-04-01T00:13:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.115+0000 7fd6e37fe640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 0x7fd6cc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.115+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6e40ad020 con 0x7fd6ec15f610 2024-04-01T00:13:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.115+0000 7fd6eaffd640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 0x7fd6cc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.116+0000 7fd6eaffd640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 0x7fd6cc06de50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d660 tx=0x7fd6dc000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.119+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd6e407b670 con 0x7fd6ec15f610 2024-04-01T00:13:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.298+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6ec0681a0 con 0x7fd6cc06b990 2024-04-01T00:13:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.305+0000 7fd6e37fe640 1 -- 172.21.15.64:0/604821922 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7fd6ec0681a0 con 0x7fd6cc06b990 2024-04-01T00:13:50.308 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:50.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 msgr2=0x7fd6cc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 0x7fd6cc06de50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d660 tx=0x7fd6dc000bc0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 msgr2=0x7fd6ec06ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009d90 tx=0x7fd6e4004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 shutdown_connections 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd6cc06b990 0x7fd6cc06de50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ec15f610 0x7fd6ec06ca20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6ec151fe0 0x7fd6ec151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 --2- 172.21.15.64:0/604821922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0cff40 0x7fd6ec150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 >> 172.21.15.64:0/604821922 conn(0x7fd6ec06b4e0 msgr2=0x7fd6ec0aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 shutdown_connections 2024-04-01T00:13:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.307+0000 7fd6f14ae640 1 -- 172.21.15.64:0/604821922 wait complete. 2024-04-01T00:13:50.319 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-01T00:13:48.746669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219288,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755432,"statfs":{"total":773094113280,"available":772869562368,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000613"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-01T00:13:38.107571+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:38.107571+0000","last_peered":"2024-04-01T00:13:38.107571+0000","last_clean":"2024-04-01T00:13:38.107571+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:38.107571+0000","last_undegraded":"2024-04-01T00:13:38.107571+0000","last_fullsized":"2024-04-01T00:13:38.107571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28372,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343468,"statfs":{"total":96636764160,"available":96607711232,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:50.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.460+0000 7fd157a8f640 1 Processor -- start 2024-04-01T00:13:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.460+0000 7fd157a8f640 1 -- start start 2024-04-01T00:13:50.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd157a8f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd15015bb70 0x7fd15015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd157a8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15006d560 0x7fd150071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd157a8f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd15015bf50 con 0x7fd150071a80 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd157a8f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd1500afe60 con 0x7fd15015bb70 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd157a8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd15015eb90 con 0x7fd15006d560 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd15582b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd15015bb70 0x7fd15015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd15602c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd150071a80 0x7fd1500c9cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42934/0 (socket says 172.21.15.64:42934) 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd15602c640 1 -- 172.21.15.64:0/1061301626 learned_addr learned my addr 172.21.15.64:0/1061301626 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.461+0000 7fd15502a640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15006d560 0x7fd150071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd15582b640 1 -- 172.21.15.64:0/1061301626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15006d560 msgr2=0x7fd150071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd15582b640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15006d560 0x7fd150071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd15582b640 1 -- 172.21.15.64:0/1061301626 >> v1:172.21.15.64:6789/0 conn(0x7fd150071a80 legacy=0x7fd1500c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd154829640 1 -- 172.21.15.64:0/1061301626 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492605511 0 0) 0x7fd15015bf50 con 0x7fd150071a80 2024-04-01T00:13:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd15582b640 1 -- 172.21.15.64:0/1061301626 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15015ee70 con 0x7fd15015bb70 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd15582b640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd15015e5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd148009d90 tx=0x7fd14802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1da0848fb478a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd154829640 1 -- 172.21.15.64:0/1061301626 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148039070 con 0x7fd15015bb70 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd154829640 1 -- 172.21.15.64:0/1061301626 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14802cc50 con 0x7fd15015bb70 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.462+0000 7fd154829640 1 -- 172.21.15.64:0/1061301626 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148034440 con 0x7fd15015bb70 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 -- 172.21.15.64:0/1061301626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 msgr2=0x7fd15015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd15015e5b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd148009d90 tx=0x7fd14802bea0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 -- 172.21.15.64:0/1061301626 shutdown_connections 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15006d560 0x7fd150071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 --2- 172.21.15.64:0/1061301626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd15015e5b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 -- 172.21.15.64:0/1061301626 >> 172.21.15.64:0/1061301626 conn(0x7fd1500d03e0 msgr2=0x7fd1500d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 -- 172.21.15.64:0/1061301626 shutdown_connections 2024-04-01T00:13:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.463+0000 7fd157a8f640 1 -- 172.21.15.64:0/1061301626 wait complete. 2024-04-01T00:13:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.464+0000 7fd157a8f640 1 Processor -- start 2024-04-01T00:13:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.464+0000 7fd157a8f640 1 -- start start 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15006d560 0x7fd1501512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd150071a80 0x7fd1501e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd15006d9c0 con 0x7fd15015bb70 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1500c97d0 con 0x7fd15006d560 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd157a8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd15015f430 con 0x7fd150071a80 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53410/0 (socket says 172.21.15.64:53410) 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 -- 172.21.15.64:0/557045993 learned_addr learned my addr 172.21.15.64:0/557045993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15502a640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd150071a80 0x7fd1501e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15582b640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15006d560 0x7fd1501512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 -- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd150071a80 msgr2=0x7fd1501e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd150071a80 0x7fd1501e6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 -- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15006d560 msgr2=0x7fd1501512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15006d560 0x7fd1501512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 -- 172.21.15.64:0/557045993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1501e6b60 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.465+0000 7fd15602c640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c440 tx=0x7fd14000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.466+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400074e0 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.466+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1400047e0 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.466+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004c10 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.466+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15015c270 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.466+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15015c830 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.467+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1500681a0 con 0x7fd15015bb70 2024-04-01T00:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.467+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd140011070 con 0x7fd15015bb70 2024-04-01T00:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.467+0000 7fd14e7fc640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 0x7fd13c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.468+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd14008a130 con 0x7fd15015bb70 2024-04-01T00:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.468+0000 7fd15582b640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 0x7fd13c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.468+0000 7fd15582b640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 0x7fd13c06de50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd148036000 tx=0x7fd148030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.472+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd140058780 con 0x7fd15015bb70 2024-04-01T00:13:50.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.652+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1501e6e00 con 0x7fd13c06b990 2024-04-01T00:13:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.658+0000 7fd14e7fc640 1 -- 172.21.15.64:0/557045993 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7fd1501e6e00 con 0x7fd13c06b990 2024-04-01T00:13:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:50.662 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:50.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 msgr2=0x7fd13c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 0x7fd13c06de50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd148036000 tx=0x7fd148030000 comp rx=0 tx=0).stop 2024-04-01T00:13:50.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 msgr2=0x7fd1501606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c440 tx=0x7fd14000c900 comp rx=0 tx=0).stop 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 shutdown_connections 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd13c06b990 0x7fd13c06de50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd15015bb70 0x7fd1501606f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd150071a80 0x7fd1501e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 --2- 172.21.15.64:0/557045993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15006d560 0x7fd1501512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 >> 172.21.15.64:0/557045993 conn(0x7fd1500d03e0 msgr2=0x7fd1500cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 shutdown_connections 2024-04-01T00:13:50.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.661+0000 7fd157a8f640 1 -- 172.21.15.64:0/557045993 wait complete. 2024-04-01T00:13:50.672 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-01T00:13:48.746669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219288,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755432,"statfs":{"total":773094113280,"available":772869562368,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000613"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-01T00:13:38.107571+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:38.107571+0000","last_peered":"2024-04-01T00:13:38.107571+0000","last_clean":"2024-04-01T00:13:38.107571+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:38.107571+0000","last_undegraded":"2024-04-01T00:13:38.107571+0000","last_fullsized":"2024-04-01T00:13:38.107571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:46.007027+0000","last_change":"2024-04-01T00:13:46.007027+0000","last_active":"2024-04-01T00:13:46.007027+0000","last_peered":"2024-04-01T00:13:46.007027+0000","last_clean":"2024-04-01T00:13:46.007027+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:46.007027+0000","last_undegraded":"2024-04-01T00:13:46.007027+0000","last_fullsized":"2024-04-01T00:13:46.007027+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-01T00:13:45.001667+0000","last_change":"2024-04-01T00:13:45.001667+0000","last_active":"2024-04-01T00:13:45.001667+0000","last_peered":"2024-04-01T00:13:45.001667+0000","last_clean":"2024-04-01T00:13:45.001667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:13:45.001667+0000","last_undegraded":"2024-04-01T00:13:45.001667+0000","last_fullsized":"2024-04-01T00:13:45.001667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28372,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343468,"statfs":{"total":96636764160,"available":96607711232,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:50.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.814+0000 7fec0d22a640 1 Processor -- start 2024-04-01T00:13:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.814+0000 7fec0d22a640 1 -- start start 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0d22a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 0x7fec080b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0d22a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec080b5070 con 0x7fec080d1030 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0d22a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec080b5350 con 0x7fec080d2740 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0d22a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec080d4e60 con 0x7fec08161b90 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0759e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fec080d1030 0x7fec080b7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42942/0 (socket says 172.21.15.64:42942) 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0759e640 1 -- 172.21.15.64:0/1937931750 learned_addr learned my addr 172.21.15.64:0/1937931750 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0659c640 1 --2- 172.21.15.64:0/1937931750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 0x7fec080b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346660815 0 0) 0x7fec080b5070 con 0x7fec080d1030 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febf0003520 con 0x7fec080d1030 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088063594 0 0) 0x7fec080b5350 con 0x7fec080d2740 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec080b5070 con 0x7fec080d2740 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0659c640 1 -- 172.21.15.64:0/1937931750 >> v1:172.21.15.100:6789/0 conn(0x7fec080d2740 legacy=0x7fec08161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0659c640 1 -- 172.21.15.64:0/1937931750 >> v1:172.21.15.64:6789/0 conn(0x7fec080d1030 legacy=0x7fec080b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.815+0000 7fec0659c640 1 -- 172.21.15.64:0/1937931750 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec080b6de0 con 0x7fec08161b90 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec0659c640 1 --2- 172.21.15.64:0/1937931750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 0x7fec080b7300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febfc009800 tx=0x7febfc02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcedee6c5dcafe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc035020 con 0x7fec08161b90 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc031890 con 0x7fec08161b90 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec05d9b640 1 -- 172.21.15.64:0/1937931750 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc031c60 con 0x7fec08161b90 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec0d22a640 1 -- 172.21.15.64:0/1937931750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 msgr2=0x7fec080b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.816+0000 7fec0d22a640 1 --2- 172.21.15.64:0/1937931750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 0x7fec080b7300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febfc009800 tx=0x7febfc02b930 comp rx=0 tx=0).stop 2024-04-01T00:13:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.817+0000 7fec0d22a640 1 -- 172.21.15.64:0/1937931750 shutdown_connections 2024-04-01T00:13:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.817+0000 7fec0d22a640 1 --2- 172.21.15.64:0/1937931750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec08161b90 0x7fec080b7300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.817+0000 7fec0d22a640 1 -- 172.21.15.64:0/1937931750 >> 172.21.15.64:0/1937931750 conn(0x7fec080d3ef0 msgr2=0x7fec080d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.817+0000 7fec0d22a640 1 -- 172.21.15.64:0/1937931750 shutdown_connections 2024-04-01T00:13:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.817+0000 7fec0d22a640 1 -- 172.21.15.64:0/1937931750 wait complete. 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.818+0000 7fec0d22a640 1 Processor -- start 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.818+0000 7fec0d22a640 1 -- start start 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec080d1030 0x7fec081f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec08161b90 0x7fec080c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec080d4b90 con 0x7fec080d2740 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec080d4a10 con 0x7fec08161b90 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0d22a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec080d1cd0 con 0x7fec080d1030 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53412/0 (socket says 172.21.15.64:53412) 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 -- 172.21.15.64:0/132064519 learned_addr learned my addr 172.21.15.64:0/132064519 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec06d9d640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec080d1030 0x7fec081f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0759e640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec08161b90 0x7fec080c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 -- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec080d1030 msgr2=0x7fec081f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec080d1030 0x7fec081f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 -- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec08161b90 msgr2=0x7fec080c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec08161b90 0x7fec080c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 -- 172.21.15.64:0/132064519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec081f4d50 con 0x7fec080d2740 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.819+0000 7fec0659c640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febfc034040 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.820+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc040070 con 0x7fec080d2740 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.820+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc030d20 con 0x7fec080d2740 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.820+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec080c7dc0 con 0x7fec080d2740 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.820+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc035040 con 0x7fec080d2740 2024-04-01T00:13:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.820+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec080c8380 con 0x7fec080d2740 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.821+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec080d3a50 con 0x7fec080d2740 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.821+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7febfc04b060 con 0x7fec080d2740 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.822+0000 7febef7fe640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 0x7febf406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.822+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febfc0ac3e0 con 0x7fec080d2740 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.822+0000 7fec06d9d640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 0x7febf406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.822+0000 7fec06d9d640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 0x7febf406de80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febf8007fd0 tx=0x7febf8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:50.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:50.826+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fec080d3a50 con 0x7fec080d2740 2024-04-01T00:13:51.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.005+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec080681a0 con 0x7febf406b9c0 2024-04-01T00:13:51.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.017+0000 7febef7fe640 1 -- 172.21.15.64:0/132064519 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7fec080681a0 con 0x7febf406b9c0 2024-04-01T00:13:51.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:51.022 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:51.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 msgr2=0x7febf406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 0x7febf406de80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febf8007fd0 tx=0x7febf8009040 comp rx=0 tx=0).stop 2024-04-01T00:13:51.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 msgr2=0x7fec081f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febfc034040 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 shutdown_connections 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf406b9c0 0x7febf406de80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec08161b90 0x7fec080c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec080d2740 0x7fec081f1230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 --2- 172.21.15.64:0/132064519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec080d1030 0x7fec081f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 >> 172.21.15.64:0/132064519 conn(0x7fec080d3ef0 msgr2=0x7fec080d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 shutdown_connections 2024-04-01T00:13:51.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.020+0000 7fec0d22a640 1 -- 172.21.15.64:0/132064519 wait complete. 2024-04-01T00:13:51.031 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-01T00:13:50.746916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.739501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905422+0000","last_change":"2024-04-01T00:13:47.045277+0000","last_active":"2024-04-01T00:13:48.905422+0000","last_peered":"2024-04-01T00:13:48.905422+0000","last_clean":"2024-04-01T00:13:48.905422+0000","last_became_active":"2024-04-01T00:13:47.044728+0000","last_became_peered":"2024-04-01T00:13:47.044728+0000","last_unstale":"2024-04-01T00:13:48.905422+0000","last_undegraded":"2024-04-01T00:13:48.905422+0000","last_fullsized":"2024-04-01T00:13:48.905422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:50:54.919900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025542+0000","last_change":"2024-04-01T00:13:47.039324+0000","last_active":"2024-04-01T00:13:48.025542+0000","last_peered":"2024-04-01T00:13:48.025542+0000","last_clean":"2024-04-01T00:13:48.025542+0000","last_became_active":"2024-04-01T00:13:47.039002+0000","last_became_peered":"2024-04-01T00:13:47.039002+0000","last_unstale":"2024-04-01T00:13:48.025542+0000","last_undegraded":"2024-04-01T00:13:48.025542+0000","last_fullsized":"2024-04-01T00:13:48.025542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906198+0000","last_change":"2024-04-01T00:13:47.039670+0000","last_active":"2024-04-01T00:13:48.906198+0000","last_peered":"2024-04-01T00:13:48.906198+0000","last_clean":"2024-04-01T00:13:48.906198+0000","last_became_active":"2024-04-01T00:13:47.037096+0000","last_became_peered":"2024-04-01T00:13:47.037096+0000","last_unstale":"2024-04-01T00:13:48.906198+0000","last_undegraded":"2024-04-01T00:13:48.906198+0000","last_fullsized":"2024-04-01T00:13:48.906198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:05:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026858+0000","last_change":"2024-04-01T00:13:47.033997+0000","last_active":"2024-04-01T00:13:48.026858+0000","last_peered":"2024-04-01T00:13:48.026858+0000","last_clean":"2024-04-01T00:13:48.026858+0000","last_became_active":"2024-04-01T00:13:47.033682+0000","last_became_peered":"2024-04-01T00:13:47.033682+0000","last_unstale":"2024-04-01T00:13:48.026858+0000","last_undegraded":"2024-04-01T00:13:48.026858+0000","last_fullsized":"2024-04-01T00:13:48.026858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905169+0000","last_change":"2024-04-01T00:13:47.040873+0000","last_active":"2024-04-01T00:13:48.905169+0000","last_peered":"2024-04-01T00:13:48.905169+0000","last_clean":"2024-04-01T00:13:48.905169+0000","last_became_active":"2024-04-01T00:13:47.039953+0000","last_became_peered":"2024-04-01T00:13:47.039953+0000","last_unstale":"2024-04-01T00:13:48.905169+0000","last_undegraded":"2024-04-01T00:13:48.905169+0000","last_fullsized":"2024-04-01T00:13:48.905169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026638+0000","last_change":"2024-04-01T00:13:47.041029+0000","last_active":"2024-04-01T00:13:48.026638+0000","last_peered":"2024-04-01T00:13:48.026638+0000","last_clean":"2024-04-01T00:13:48.026638+0000","last_became_active":"2024-04-01T00:13:47.040771+0000","last_became_peered":"2024-04-01T00:13:47.040771+0000","last_unstale":"2024-04-01T00:13:48.026638+0000","last_undegraded":"2024-04-01T00:13:48.026638+0000","last_fullsized":"2024-04-01T00:13:48.026638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904941+0000","last_change":"2024-04-01T00:13:47.042649+0000","last_active":"2024-04-01T00:13:48.904941+0000","last_peered":"2024-04-01T00:13:48.904941+0000","last_clean":"2024-04-01T00:13:48.904941+0000","last_became_active":"2024-04-01T00:13:47.042373+0000","last_became_peered":"2024-04-01T00:13:47.042373+0000","last_unstale":"2024-04-01T00:13:48.904941+0000","last_undegraded":"2024-04-01T00:13:48.904941+0000","last_fullsized":"2024-04-01T00:13:48.904941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905488+0000","last_change":"2024-04-01T00:13:47.031258+0000","last_active":"2024-04-01T00:13:48.905488+0000","last_peered":"2024-04-01T00:13:48.905488+0000","last_clean":"2024-04-01T00:13:48.905488+0000","last_became_active":"2024-04-01T00:13:47.030972+0000","last_became_peered":"2024-04-01T00:13:47.030972+0000","last_unstale":"2024-04-01T00:13:48.905488+0000","last_undegraded":"2024-04-01T00:13:48.905488+0000","last_fullsized":"2024-04-01T00:13:48.905488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905452+0000","last_change":"2024-04-01T00:13:47.045417+0000","last_active":"2024-04-01T00:13:48.905452+0000","last_peered":"2024-04-01T00:13:48.905452+0000","last_clean":"2024-04-01T00:13:48.905452+0000","last_became_active":"2024-04-01T00:13:47.044978+0000","last_became_peered":"2024-04-01T00:13:47.044978+0000","last_unstale":"2024-04-01T00:13:48.905452+0000","last_undegraded":"2024-04-01T00:13:48.905452+0000","last_fullsized":"2024-04-01T00:13:48.905452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:12:06.404460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030825+0000","last_change":"2024-04-01T00:13:47.035602+0000","last_active":"2024-04-01T00:13:48.030825+0000","last_peered":"2024-04-01T00:13:48.030825+0000","last_clean":"2024-04-01T00:13:48.030825+0000","last_became_active":"2024-04-01T00:13:47.035293+0000","last_became_peered":"2024-04-01T00:13:47.035293+0000","last_unstale":"2024-04-01T00:13:48.030825+0000","last_undegraded":"2024-04-01T00:13:48.030825+0000","last_fullsized":"2024-04-01T00:13:48.030825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030712+0000","last_change":"2024-04-01T00:13:47.037792+0000","last_active":"2024-04-01T00:13:48.030712+0000","last_peered":"2024-04-01T00:13:48.030712+0000","last_clean":"2024-04-01T00:13:48.030712+0000","last_became_active":"2024-04-01T00:13:47.037508+0000","last_became_peered":"2024-04-01T00:13:47.037508+0000","last_unstale":"2024-04-01T00:13:48.030712+0000","last_undegraded":"2024-04-01T00:13:48.030712+0000","last_fullsized":"2024-04-01T00:13:48.030712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028123+0000","last_change":"2024-04-01T00:13:47.026838+0000","last_active":"2024-04-01T00:13:48.028123+0000","last_peered":"2024-04-01T00:13:48.028123+0000","last_clean":"2024-04-01T00:13:48.028123+0000","last_became_active":"2024-04-01T00:13:47.026586+0000","last_became_peered":"2024-04-01T00:13:47.026586+0000","last_unstale":"2024-04-01T00:13:48.028123+0000","last_undegraded":"2024-04-01T00:13:48.028123+0000","last_fullsized":"2024-04-01T00:13:48.028123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:13.513798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905964+0000","last_change":"2024-04-01T00:13:47.028848+0000","last_active":"2024-04-01T00:13:48.905964+0000","last_peered":"2024-04-01T00:13:48.905964+0000","last_clean":"2024-04-01T00:13:48.905964+0000","last_became_active":"2024-04-01T00:13:47.028577+0000","last_became_peered":"2024-04-01T00:13:47.028577+0000","last_unstale":"2024-04-01T00:13:48.905964+0000","last_undegraded":"2024-04-01T00:13:48.905964+0000","last_fullsized":"2024-04-01T00:13:48.905964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906018+0000","last_change":"2024-04-01T00:13:47.039847+0000","last_active":"2024-04-01T00:13:48.906018+0000","last_peered":"2024-04-01T00:13:48.906018+0000","last_clean":"2024-04-01T00:13:48.906018+0000","last_became_active":"2024-04-01T00:13:47.039459+0000","last_became_peered":"2024-04-01T00:13:47.039459+0000","last_unstale":"2024-04-01T00:13:48.906018+0000","last_undegraded":"2024-04-01T00:13:48.906018+0000","last_fullsized":"2024-04-01T00:13:48.906018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:51.654019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025413+0000","last_change":"2024-04-01T00:13:47.034288+0000","last_active":"2024-04-01T00:13:48.025413+0000","last_peered":"2024-04-01T00:13:48.025413+0000","last_clean":"2024-04-01T00:13:48.025413+0000","last_became_active":"2024-04-01T00:13:47.034022+0000","last_became_peered":"2024-04-01T00:13:47.034022+0000","last_unstale":"2024-04-01T00:13:48.025413+0000","last_undegraded":"2024-04-01T00:13:48.025413+0000","last_fullsized":"2024-04-01T00:13:48.025413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026557+0000","last_change":"2024-04-01T00:13:47.039173+0000","last_active":"2024-04-01T00:13:48.026557+0000","last_peered":"2024-04-01T00:13:48.026557+0000","last_clean":"2024-04-01T00:13:48.026557+0000","last_became_active":"2024-04-01T00:13:47.038845+0000","last_became_peered":"2024-04-01T00:13:47.038845+0000","last_unstale":"2024-04-01T00:13:48.026557+0000","last_undegraded":"2024-04-01T00:13:48.026557+0000","last_fullsized":"2024-04-01T00:13:48.026557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027411+0000","last_change":"2024-04-01T00:13:47.043400+0000","last_active":"2024-04-01T00:13:48.027411+0000","last_peered":"2024-04-01T00:13:48.027411+0000","last_clean":"2024-04-01T00:13:48.027411+0000","last_became_active":"2024-04-01T00:13:47.043120+0000","last_became_peered":"2024-04-01T00:13:47.043120+0000","last_unstale":"2024-04-01T00:13:48.027411+0000","last_undegraded":"2024-04-01T00:13:48.027411+0000","last_fullsized":"2024-04-01T00:13:48.027411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905429+0000","last_change":"2024-04-01T00:13:47.035436+0000","last_active":"2024-04-01T00:13:48.905429+0000","last_peered":"2024-04-01T00:13:48.905429+0000","last_clean":"2024-04-01T00:13:48.905429+0000","last_became_active":"2024-04-01T00:13:47.035072+0000","last_became_peered":"2024-04-01T00:13:47.035072+0000","last_unstale":"2024-04-01T00:13:48.905429+0000","last_undegraded":"2024-04-01T00:13:48.905429+0000","last_fullsized":"2024-04-01T00:13:48.905429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905379+0000","last_change":"2024-04-01T00:13:47.040506+0000","last_active":"2024-04-01T00:13:48.905379+0000","last_peered":"2024-04-01T00:13:48.905379+0000","last_clean":"2024-04-01T00:13:48.905379+0000","last_became_active":"2024-04-01T00:13:47.040244+0000","last_became_peered":"2024-04-01T00:13:47.040244+0000","last_unstale":"2024-04-01T00:13:48.905379+0000","last_undegraded":"2024-04-01T00:13:48.905379+0000","last_fullsized":"2024-04-01T00:13:48.905379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030707+0000","last_change":"2024-04-01T00:13:47.031532+0000","last_active":"2024-04-01T00:13:48.030707+0000","last_peered":"2024-04-01T00:13:48.030707+0000","last_clean":"2024-04-01T00:13:48.030707+0000","last_became_active":"2024-04-01T00:13:47.031252+0000","last_became_peered":"2024-04-01T00:13:47.031252+0000","last_unstale":"2024-04-01T00:13:48.030707+0000","last_undegraded":"2024-04-01T00:13:48.030707+0000","last_fullsized":"2024-04-01T00:13:48.030707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906466+0000","last_change":"2024-04-01T00:13:47.047404+0000","last_active":"2024-04-01T00:13:48.906466+0000","last_peered":"2024-04-01T00:13:48.906466+0000","last_clean":"2024-04-01T00:13:48.906466+0000","last_became_active":"2024-04-01T00:13:47.047080+0000","last_became_peered":"2024-04-01T00:13:47.047080+0000","last_unstale":"2024-04-01T00:13:48.906466+0000","last_undegraded":"2024-04-01T00:13:48.906466+0000","last_fullsized":"2024-04-01T00:13:48.906466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026698+0000","last_change":"2024-04-01T00:13:47.033044+0000","last_active":"2024-04-01T00:13:48.026698+0000","last_peered":"2024-04-01T00:13:48.026698+0000","last_clean":"2024-04-01T00:13:48.026698+0000","last_became_active":"2024-04-01T00:13:47.032783+0000","last_became_peered":"2024-04-01T00:13:47.032783+0000","last_unstale":"2024-04-01T00:13:48.026698+0000","last_undegraded":"2024-04-01T00:13:48.026698+0000","last_fullsized":"2024-04-01T00:13:48.026698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905485+0000","last_change":"2024-04-01T00:13:47.035282+0000","last_active":"2024-04-01T00:13:48.905485+0000","last_peered":"2024-04-01T00:13:48.905485+0000","last_clean":"2024-04-01T00:13:48.905485+0000","last_became_active":"2024-04-01T00:13:47.034953+0000","last_became_peered":"2024-04-01T00:13:47.034953+0000","last_unstale":"2024-04-01T00:13:48.905485+0000","last_undegraded":"2024-04-01T00:13:48.905485+0000","last_fullsized":"2024-04-01T00:13:48.905485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026661+0000","last_change":"2024-04-01T00:13:47.034177+0000","last_active":"2024-04-01T00:13:48.026661+0000","last_peered":"2024-04-01T00:13:48.026661+0000","last_clean":"2024-04-01T00:13:48.026661+0000","last_became_active":"2024-04-01T00:13:47.033426+0000","last_became_peered":"2024-04-01T00:13:47.033426+0000","last_unstale":"2024-04-01T00:13:48.026661+0000","last_undegraded":"2024-04-01T00:13:48.026661+0000","last_fullsized":"2024-04-01T00:13:48.026661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906196+0000","last_change":"2024-04-01T00:13:47.034202+0000","last_active":"2024-04-01T00:13:48.906196+0000","last_peered":"2024-04-01T00:13:48.906196+0000","last_clean":"2024-04-01T00:13:48.906196+0000","last_became_active":"2024-04-01T00:13:47.033941+0000","last_became_peered":"2024-04-01T00:13:47.033941+0000","last_unstale":"2024-04-01T00:13:48.906196+0000","last_undegraded":"2024-04-01T00:13:48.906196+0000","last_fullsized":"2024-04-01T00:13:48.906196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030886+0000","last_change":"2024-04-01T00:13:47.042844+0000","last_active":"2024-04-01T00:13:48.030886+0000","last_peered":"2024-04-01T00:13:48.030886+0000","last_clean":"2024-04-01T00:13:48.030886+0000","last_became_active":"2024-04-01T00:13:47.042518+0000","last_became_peered":"2024-04-01T00:13:47.042518+0000","last_unstale":"2024-04-01T00:13:48.030886+0000","last_undegraded":"2024-04-01T00:13:48.030886+0000","last_fullsized":"2024-04-01T00:13:48.030886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025478+0000","last_change":"2024-04-01T00:13:47.032968+0000","last_active":"2024-04-01T00:13:48.025478+0000","last_peered":"2024-04-01T00:13:48.025478+0000","last_clean":"2024-04-01T00:13:48.025478+0000","last_became_active":"2024-04-01T00:13:47.032695+0000","last_became_peered":"2024-04-01T00:13:47.032695+0000","last_unstale":"2024-04-01T00:13:48.025478+0000","last_undegraded":"2024-04-01T00:13:48.025478+0000","last_fullsized":"2024-04-01T00:13:48.025478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906706+0000","last_change":"2024-04-01T00:13:47.035447+0000","last_active":"2024-04-01T00:13:48.906706+0000","last_peered":"2024-04-01T00:13:48.906706+0000","last_clean":"2024-04-01T00:13:48.906706+0000","last_became_active":"2024-04-01T00:13:47.035003+0000","last_became_peered":"2024-04-01T00:13:47.035003+0000","last_unstale":"2024-04-01T00:13:48.906706+0000","last_undegraded":"2024-04-01T00:13:48.906706+0000","last_fullsized":"2024-04-01T00:13:48.906706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906541+0000","last_change":"2024-04-01T00:13:47.039218+0000","last_active":"2024-04-01T00:13:48.906541+0000","last_peered":"2024-04-01T00:13:48.906541+0000","last_clean":"2024-04-01T00:13:48.906541+0000","last_became_active":"2024-04-01T00:13:47.037984+0000","last_became_peered":"2024-04-01T00:13:47.037984+0000","last_unstale":"2024-04-01T00:13:48.906541+0000","last_undegraded":"2024-04-01T00:13:48.906541+0000","last_fullsized":"2024-04-01T00:13:48.906541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026792+0000","last_change":"2024-04-01T00:13:47.034521+0000","last_active":"2024-04-01T00:13:48.026792+0000","last_peered":"2024-04-01T00:13:48.026792+0000","last_clean":"2024-04-01T00:13:48.026792+0000","last_became_active":"2024-04-01T00:13:47.034251+0000","last_became_peered":"2024-04-01T00:13:47.034251+0000","last_unstale":"2024-04-01T00:13:48.026792+0000","last_undegraded":"2024-04-01T00:13:48.026792+0000","last_fullsized":"2024-04-01T00:13:48.026792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905386+0000","last_change":"2024-04-01T00:13:47.035817+0000","last_active":"2024-04-01T00:13:48.905386+0000","last_peered":"2024-04-01T00:13:48.905386+0000","last_clean":"2024-04-01T00:13:48.905386+0000","last_became_active":"2024-04-01T00:13:47.034834+0000","last_became_peered":"2024-04-01T00:13:47.034834+0000","last_unstale":"2024-04-01T00:13:48.905386+0000","last_undegraded":"2024-04-01T00:13:48.905386+0000","last_fullsized":"2024-04-01T00:13:48.905386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030497+0000","last_change":"2024-04-01T00:13:47.039829+0000","last_active":"2024-04-01T00:13:48.030497+0000","last_peered":"2024-04-01T00:13:48.030497+0000","last_clean":"2024-04-01T00:13:48.030497+0000","last_became_active":"2024-04-01T00:13:47.039517+0000","last_became_peered":"2024-04-01T00:13:47.039517+0000","last_unstale":"2024-04-01T00:13:48.030497+0000","last_undegraded":"2024-04-01T00:13:48.030497+0000","last_fullsized":"2024-04-01T00:13:48.030497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027883+0000","last_change":"2024-04-01T00:13:47.026124+0000","last_active":"2024-04-01T00:13:48.027883+0000","last_peered":"2024-04-01T00:13:48.027883+0000","last_clean":"2024-04-01T00:13:48.027883+0000","last_became_active":"2024-04-01T00:13:47.025855+0000","last_became_peered":"2024-04-01T00:13:47.025855+0000","last_unstale":"2024-04-01T00:13:48.027883+0000","last_undegraded":"2024-04-01T00:13:48.027883+0000","last_fullsized":"2024-04-01T00:13:48.027883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:56.178747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906229+0000","last_change":"2024-04-01T00:13:46.037694+0000","last_active":"2024-04-01T00:13:48.906229+0000","last_peered":"2024-04-01T00:13:48.906229+0000","last_clean":"2024-04-01T00:13:48.906229+0000","last_became_active":"2024-04-01T00:13:46.037390+0000","last_became_peered":"2024-04-01T00:13:46.037390+0000","last_unstale":"2024-04-01T00:13:48.906229+0000","last_undegraded":"2024-04-01T00:13:48.906229+0000","last_fullsized":"2024-04-01T00:13:48.906229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906550+0000","last_change":"2024-04-01T00:13:47.044544+0000","last_active":"2024-04-01T00:13:48.906550+0000","last_peered":"2024-04-01T00:13:48.906550+0000","last_clean":"2024-04-01T00:13:48.906550+0000","last_became_active":"2024-04-01T00:13:47.044266+0000","last_became_peered":"2024-04-01T00:13:47.044266+0000","last_unstale":"2024-04-01T00:13:48.906550+0000","last_undegraded":"2024-04-01T00:13:48.906550+0000","last_fullsized":"2024-04-01T00:13:48.906550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030621+0000","last_change":"2024-04-01T00:13:46.036332+0000","last_active":"2024-04-01T00:13:48.030621+0000","last_peered":"2024-04-01T00:13:48.030621+0000","last_clean":"2024-04-01T00:13:48.030621+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.030621+0000","last_undegraded":"2024-04-01T00:13:48.030621+0000","last_fullsized":"2024-04-01T00:13:48.030621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905834+0000","last_change":"2024-04-01T00:13:46.032078+0000","last_active":"2024-04-01T00:13:48.905834+0000","last_peered":"2024-04-01T00:13:48.905834+0000","last_clean":"2024-04-01T00:13:48.905834+0000","last_became_active":"2024-04-01T00:13:46.031705+0000","last_became_peered":"2024-04-01T00:13:46.031705+0000","last_unstale":"2024-04-01T00:13:48.905834+0000","last_undegraded":"2024-04-01T00:13:48.905834+0000","last_fullsized":"2024-04-01T00:13:48.905834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905625+0000","last_change":"2024-04-01T00:13:47.041444+0000","last_active":"2024-04-01T00:13:48.905625+0000","last_peered":"2024-04-01T00:13:48.905625+0000","last_clean":"2024-04-01T00:13:48.905625+0000","last_became_active":"2024-04-01T00:13:47.041163+0000","last_became_peered":"2024-04-01T00:13:47.041163+0000","last_unstale":"2024-04-01T00:13:48.905625+0000","last_undegraded":"2024-04-01T00:13:48.905625+0000","last_fullsized":"2024-04-01T00:13:48.905625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026649+0000","last_change":"2024-04-01T00:13:46.030376+0000","last_active":"2024-04-01T00:13:48.026649+0000","last_peered":"2024-04-01T00:13:48.026649+0000","last_clean":"2024-04-01T00:13:48.026649+0000","last_became_active":"2024-04-01T00:13:46.030085+0000","last_became_peered":"2024-04-01T00:13:46.030085+0000","last_unstale":"2024-04-01T00:13:48.026649+0000","last_undegraded":"2024-04-01T00:13:48.026649+0000","last_fullsized":"2024-04-01T00:13:48.026649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025414+0000","last_change":"2024-04-01T00:13:47.032819+0000","last_active":"2024-04-01T00:13:48.025414+0000","last_peered":"2024-04-01T00:13:48.025414+0000","last_clean":"2024-04-01T00:13:48.025414+0000","last_became_active":"2024-04-01T00:13:47.032501+0000","last_became_peered":"2024-04-01T00:13:47.032501+0000","last_unstale":"2024-04-01T00:13:48.025414+0000","last_undegraded":"2024-04-01T00:13:48.025414+0000","last_fullsized":"2024-04-01T00:13:48.025414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030703+0000","last_change":"2024-04-01T00:13:46.030622+0000","last_active":"2024-04-01T00:13:48.030703+0000","last_peered":"2024-04-01T00:13:48.030703+0000","last_clean":"2024-04-01T00:13:48.030703+0000","last_became_active":"2024-04-01T00:13:46.030340+0000","last_became_peered":"2024-04-01T00:13:46.030340+0000","last_unstale":"2024-04-01T00:13:48.030703+0000","last_undegraded":"2024-04-01T00:13:48.030703+0000","last_fullsized":"2024-04-01T00:13:48.030703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027187+0000","last_change":"2024-04-01T00:13:47.042195+0000","last_active":"2024-04-01T00:13:48.027187+0000","last_peered":"2024-04-01T00:13:48.027187+0000","last_clean":"2024-04-01T00:13:48.027187+0000","last_became_active":"2024-04-01T00:13:47.041925+0000","last_became_peered":"2024-04-01T00:13:47.041925+0000","last_unstale":"2024-04-01T00:13:48.027187+0000","last_undegraded":"2024-04-01T00:13:48.027187+0000","last_fullsized":"2024-04-01T00:13:48.027187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027178+0000","last_change":"2024-04-01T00:13:46.029157+0000","last_active":"2024-04-01T00:13:48.027178+0000","last_peered":"2024-04-01T00:13:48.027178+0000","last_clean":"2024-04-01T00:13:48.027178+0000","last_became_active":"2024-04-01T00:13:46.028917+0000","last_became_peered":"2024-04-01T00:13:46.028917+0000","last_unstale":"2024-04-01T00:13:48.027178+0000","last_undegraded":"2024-04-01T00:13:48.027178+0000","last_fullsized":"2024-04-01T00:13:48.027178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906652+0000","last_change":"2024-04-01T00:13:47.040888+0000","last_active":"2024-04-01T00:13:48.906652+0000","last_peered":"2024-04-01T00:13:48.906652+0000","last_clean":"2024-04-01T00:13:48.906652+0000","last_became_active":"2024-04-01T00:13:47.040621+0000","last_became_peered":"2024-04-01T00:13:47.040621+0000","last_unstale":"2024-04-01T00:13:48.906652+0000","last_undegraded":"2024-04-01T00:13:48.906652+0000","last_fullsized":"2024-04-01T00:13:48.906652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025420+0000","last_change":"2024-04-01T00:13:46.030545+0000","last_active":"2024-04-01T00:13:48.025420+0000","last_peered":"2024-04-01T00:13:48.025420+0000","last_clean":"2024-04-01T00:13:48.025420+0000","last_became_active":"2024-04-01T00:13:46.030232+0000","last_became_peered":"2024-04-01T00:13:46.030232+0000","last_unstale":"2024-04-01T00:13:48.025420+0000","last_undegraded":"2024-04-01T00:13:48.025420+0000","last_fullsized":"2024-04-01T00:13:48.025420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028260+0000","last_change":"2024-04-01T00:13:47.042059+0000","last_active":"2024-04-01T00:13:48.028260+0000","last_peered":"2024-04-01T00:13:48.028260+0000","last_clean":"2024-04-01T00:13:48.028260+0000","last_became_active":"2024-04-01T00:13:47.040047+0000","last_became_peered":"2024-04-01T00:13:47.040047+0000","last_unstale":"2024-04-01T00:13:48.028260+0000","last_undegraded":"2024-04-01T00:13:48.028260+0000","last_fullsized":"2024-04-01T00:13:48.028260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030779+0000","last_change":"2024-04-01T00:13:46.039366+0000","last_active":"2024-04-01T00:13:48.030779+0000","last_peered":"2024-04-01T00:13:48.030779+0000","last_clean":"2024-04-01T00:13:48.030779+0000","last_became_active":"2024-04-01T00:13:46.039132+0000","last_became_peered":"2024-04-01T00:13:46.039132+0000","last_unstale":"2024-04-01T00:13:48.030779+0000","last_undegraded":"2024-04-01T00:13:48.030779+0000","last_fullsized":"2024-04-01T00:13:48.030779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905125+0000","last_change":"2024-04-01T00:13:47.039118+0000","last_active":"2024-04-01T00:13:48.905125+0000","last_peered":"2024-04-01T00:13:48.905125+0000","last_clean":"2024-04-01T00:13:48.905125+0000","last_became_active":"2024-04-01T00:13:47.035193+0000","last_became_peered":"2024-04-01T00:13:47.035193+0000","last_unstale":"2024-04-01T00:13:48.905125+0000","last_undegraded":"2024-04-01T00:13:48.905125+0000","last_fullsized":"2024-04-01T00:13:48.905125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906606+0000","last_change":"2024-04-01T00:13:46.033537+0000","last_active":"2024-04-01T00:13:48.906606+0000","last_peered":"2024-04-01T00:13:48.906606+0000","last_clean":"2024-04-01T00:13:48.906606+0000","last_became_active":"2024-04-01T00:13:46.033174+0000","last_became_peered":"2024-04-01T00:13:46.033174+0000","last_unstale":"2024-04-01T00:13:48.906606+0000","last_undegraded":"2024-04-01T00:13:48.906606+0000","last_fullsized":"2024-04-01T00:13:48.906606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:15:18.515585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906163+0000","last_change":"2024-04-01T00:13:47.046668+0000","last_active":"2024-04-01T00:13:48.906163+0000","last_peered":"2024-04-01T00:13:48.906163+0000","last_clean":"2024-04-01T00:13:48.906163+0000","last_became_active":"2024-04-01T00:13:47.046387+0000","last_became_peered":"2024-04-01T00:13:47.046387+0000","last_unstale":"2024-04-01T00:13:48.906163+0000","last_undegraded":"2024-04-01T00:13:48.906163+0000","last_fullsized":"2024-04-01T00:13:48.906163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:44:37.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026803+0000","last_change":"2024-04-01T00:13:46.026873+0000","last_active":"2024-04-01T00:13:48.026803+0000","last_peered":"2024-04-01T00:13:48.026803+0000","last_clean":"2024-04-01T00:13:48.026803+0000","last_became_active":"2024-04-01T00:13:46.026564+0000","last_became_peered":"2024-04-01T00:13:46.026564+0000","last_unstale":"2024-04-01T00:13:48.026803+0000","last_undegraded":"2024-04-01T00:13:48.026803+0000","last_fullsized":"2024-04-01T00:13:48.026803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905168+0000","last_change":"2024-04-01T00:13:47.040797+0000","last_active":"2024-04-01T00:13:48.905168+0000","last_peered":"2024-04-01T00:13:48.905168+0000","last_clean":"2024-04-01T00:13:48.905168+0000","last_became_active":"2024-04-01T00:13:47.040224+0000","last_became_peered":"2024-04-01T00:13:47.040224+0000","last_unstale":"2024-04-01T00:13:48.905168+0000","last_undegraded":"2024-04-01T00:13:48.905168+0000","last_fullsized":"2024-04-01T00:13:48.905168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030476+0000","last_change":"2024-04-01T00:13:46.036254+0000","last_active":"2024-04-01T00:13:48.030476+0000","last_peered":"2024-04-01T00:13:48.030476+0000","last_clean":"2024-04-01T00:13:48.030476+0000","last_became_active":"2024-04-01T00:13:46.035969+0000","last_became_peered":"2024-04-01T00:13:46.035969+0000","last_unstale":"2024-04-01T00:13:48.030476+0000","last_undegraded":"2024-04-01T00:13:48.030476+0000","last_fullsized":"2024-04-01T00:13:48.030476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030469+0000","last_change":"2024-04-01T00:13:47.039818+0000","last_active":"2024-04-01T00:13:48.030469+0000","last_peered":"2024-04-01T00:13:48.030469+0000","last_clean":"2024-04-01T00:13:48.030469+0000","last_became_active":"2024-04-01T00:13:47.039157+0000","last_became_peered":"2024-04-01T00:13:47.039157+0000","last_unstale":"2024-04-01T00:13:48.030469+0000","last_undegraded":"2024-04-01T00:13:48.030469+0000","last_fullsized":"2024-04-01T00:13:48.030469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905588+0000","last_change":"2024-04-01T00:13:46.023660+0000","last_active":"2024-04-01T00:13:48.905588+0000","last_peered":"2024-04-01T00:13:48.905588+0000","last_clean":"2024-04-01T00:13:48.905588+0000","last_became_active":"2024-04-01T00:13:46.023368+0000","last_became_peered":"2024-04-01T00:13:46.023368+0000","last_unstale":"2024-04-01T00:13:48.905588+0000","last_undegraded":"2024-04-01T00:13:48.905588+0000","last_fullsized":"2024-04-01T00:13:48.905588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:11:34.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904995+0000","last_change":"2024-04-01T00:13:47.040468+0000","last_active":"2024-04-01T00:13:48.904995+0000","last_peered":"2024-04-01T00:13:48.904995+0000","last_clean":"2024-04-01T00:13:48.904995+0000","last_became_active":"2024-04-01T00:13:47.040198+0000","last_became_peered":"2024-04-01T00:13:47.040198+0000","last_unstale":"2024-04-01T00:13:48.904995+0000","last_undegraded":"2024-04-01T00:13:48.904995+0000","last_fullsized":"2024-04-01T00:13:48.904995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028251+0000","last_change":"2024-04-01T00:13:46.029828+0000","last_active":"2024-04-01T00:13:48.028251+0000","last_peered":"2024-04-01T00:13:48.028251+0000","last_clean":"2024-04-01T00:13:48.028251+0000","last_became_active":"2024-04-01T00:13:46.029514+0000","last_became_peered":"2024-04-01T00:13:46.029514+0000","last_unstale":"2024-04-01T00:13:48.028251+0000","last_undegraded":"2024-04-01T00:13:48.028251+0000","last_fullsized":"2024-04-01T00:13:48.028251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:16:56.432398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905327+0000","last_change":"2024-04-01T00:13:47.037521+0000","last_active":"2024-04-01T00:13:48.905327+0000","last_peered":"2024-04-01T00:13:48.905327+0000","last_clean":"2024-04-01T00:13:48.905327+0000","last_became_active":"2024-04-01T00:13:47.037208+0000","last_became_peered":"2024-04-01T00:13:47.037208+0000","last_unstale":"2024-04-01T00:13:48.905327+0000","last_undegraded":"2024-04-01T00:13:48.905327+0000","last_fullsized":"2024-04-01T00:13:48.905327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905466+0000","last_change":"2024-04-01T00:13:46.029659+0000","last_active":"2024-04-01T00:13:48.905466+0000","last_peered":"2024-04-01T00:13:48.905466+0000","last_clean":"2024-04-01T00:13:48.905466+0000","last_became_active":"2024-04-01T00:13:46.029376+0000","last_became_peered":"2024-04-01T00:13:46.029376+0000","last_unstale":"2024-04-01T00:13:48.905466+0000","last_undegraded":"2024-04-01T00:13:48.905466+0000","last_fullsized":"2024-04-01T00:13:48.905466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905503+0000","last_change":"2024-04-01T00:13:47.045515+0000","last_active":"2024-04-01T00:13:48.905503+0000","last_peered":"2024-04-01T00:13:48.905503+0000","last_clean":"2024-04-01T00:13:48.905503+0000","last_became_active":"2024-04-01T00:13:47.045138+0000","last_became_peered":"2024-04-01T00:13:47.045138+0000","last_unstale":"2024-04-01T00:13:48.905503+0000","last_undegraded":"2024-04-01T00:13:48.905503+0000","last_fullsized":"2024-04-01T00:13:48.905503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:32:58.179314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025530+0000","last_change":"2024-04-01T00:13:46.033447+0000","last_active":"2024-04-01T00:13:48.025530+0000","last_peered":"2024-04-01T00:13:48.025530+0000","last_clean":"2024-04-01T00:13:48.025530+0000","last_became_active":"2024-04-01T00:13:46.033126+0000","last_became_peered":"2024-04-01T00:13:46.033126+0000","last_unstale":"2024-04-01T00:13:48.025530+0000","last_undegraded":"2024-04-01T00:13:48.025530+0000","last_fullsized":"2024-04-01T00:13:48.025530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027072+0000","last_change":"2024-04-01T00:13:47.033643+0000","last_active":"2024-04-01T00:13:48.027072+0000","last_peered":"2024-04-01T00:13:48.027072+0000","last_clean":"2024-04-01T00:13:48.027072+0000","last_became_active":"2024-04-01T00:13:47.033350+0000","last_became_peered":"2024-04-01T00:13:47.033350+0000","last_unstale":"2024-04-01T00:13:48.027072+0000","last_undegraded":"2024-04-01T00:13:48.027072+0000","last_fullsized":"2024-04-01T00:13:48.027072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905480+0000","last_change":"2024-04-01T00:13:46.021048+0000","last_active":"2024-04-01T00:13:48.905480+0000","last_peered":"2024-04-01T00:13:48.905480+0000","last_clean":"2024-04-01T00:13:48.905480+0000","last_became_active":"2024-04-01T00:13:46.020612+0000","last_became_peered":"2024-04-01T00:13:46.020612+0000","last_unstale":"2024-04-01T00:13:48.905480+0000","last_undegraded":"2024-04-01T00:13:48.905480+0000","last_fullsized":"2024-04-01T00:13:48.905480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905112+0000","last_change":"2024-04-01T00:13:47.038059+0000","last_active":"2024-04-01T00:13:48.905112+0000","last_peered":"2024-04-01T00:13:48.905112+0000","last_clean":"2024-04-01T00:13:48.905112+0000","last_became_active":"2024-04-01T00:13:47.034213+0000","last_became_peered":"2024-04-01T00:13:47.034213+0000","last_unstale":"2024-04-01T00:13:48.905112+0000","last_undegraded":"2024-04-01T00:13:48.905112+0000","last_fullsized":"2024-04-01T00:13:48.905112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904882+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:48.904882+0000","last_peered":"2024-04-01T00:13:48.904882+0000","last_clean":"2024-04-01T00:13:48.904882+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:48.904882+0000","last_undegraded":"2024-04-01T00:13:48.904882+0000","last_fullsized":"2024-04-01T00:13:48.904882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026668+0000","last_change":"2024-04-01T00:13:46.019635+0000","last_active":"2024-04-01T00:13:48.026668+0000","last_peered":"2024-04-01T00:13:48.026668+0000","last_clean":"2024-04-01T00:13:48.026668+0000","last_became_active":"2024-04-01T00:13:46.019355+0000","last_became_peered":"2024-04-01T00:13:46.019355+0000","last_unstale":"2024-04-01T00:13:48.026668+0000","last_undegraded":"2024-04-01T00:13:48.026668+0000","last_fullsized":"2024-04-01T00:13:48.026668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906283+0000","last_change":"2024-04-01T00:13:47.034055+0000","last_active":"2024-04-01T00:13:48.906283+0000","last_peered":"2024-04-01T00:13:48.906283+0000","last_clean":"2024-04-01T00:13:48.906283+0000","last_became_active":"2024-04-01T00:13:47.033778+0000","last_became_peered":"2024-04-01T00:13:47.033778+0000","last_unstale":"2024-04-01T00:13:48.906283+0000","last_undegraded":"2024-04-01T00:13:48.906283+0000","last_fullsized":"2024-04-01T00:13:48.906283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904818+0000","last_change":"2024-04-01T00:13:46.037663+0000","last_active":"2024-04-01T00:13:48.904818+0000","last_peered":"2024-04-01T00:13:48.904818+0000","last_clean":"2024-04-01T00:13:48.904818+0000","last_became_active":"2024-04-01T00:13:46.037388+0000","last_became_peered":"2024-04-01T00:13:46.037388+0000","last_unstale":"2024-04-01T00:13:48.904818+0000","last_undegraded":"2024-04-01T00:13:48.904818+0000","last_fullsized":"2024-04-01T00:13:48.904818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906330+0000","last_change":"2024-04-01T00:13:47.038068+0000","last_active":"2024-04-01T00:13:48.906330+0000","last_peered":"2024-04-01T00:13:48.906330+0000","last_clean":"2024-04-01T00:13:48.906330+0000","last_became_active":"2024-04-01T00:13:47.037774+0000","last_became_peered":"2024-04-01T00:13:47.037774+0000","last_unstale":"2024-04-01T00:13:48.906330+0000","last_undegraded":"2024-04-01T00:13:48.906330+0000","last_fullsized":"2024-04-01T00:13:48.906330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:17:27.372297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905353+0000","last_change":"2024-04-01T00:13:46.029621+0000","last_active":"2024-04-01T00:13:48.905353+0000","last_peered":"2024-04-01T00:13:48.905353+0000","last_clean":"2024-04-01T00:13:48.905353+0000","last_became_active":"2024-04-01T00:13:46.029319+0000","last_became_peered":"2024-04-01T00:13:46.029319+0000","last_unstale":"2024-04-01T00:13:48.905353+0000","last_undegraded":"2024-04-01T00:13:48.905353+0000","last_fullsized":"2024-04-01T00:13:48.905353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025440+0000","last_change":"2024-04-01T00:13:47.030092+0000","last_active":"2024-04-01T00:13:48.025440+0000","last_peered":"2024-04-01T00:13:48.025440+0000","last_clean":"2024-04-01T00:13:48.025440+0000","last_became_active":"2024-04-01T00:13:47.029837+0000","last_became_peered":"2024-04-01T00:13:47.029837+0000","last_unstale":"2024-04-01T00:13:48.025440+0000","last_undegraded":"2024-04-01T00:13:48.025440+0000","last_fullsized":"2024-04-01T00:13:48.025440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026885+0000","last_change":"2024-04-01T00:13:46.027134+0000","last_active":"2024-04-01T00:13:48.026885+0000","last_peered":"2024-04-01T00:13:48.026885+0000","last_clean":"2024-04-01T00:13:48.026885+0000","last_became_active":"2024-04-01T00:13:46.026855+0000","last_became_peered":"2024-04-01T00:13:46.026855+0000","last_unstale":"2024-04-01T00:13:48.026885+0000","last_undegraded":"2024-04-01T00:13:48.026885+0000","last_fullsized":"2024-04-01T00:13:48.026885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028071+0000","last_change":"2024-04-01T00:13:47.041868+0000","last_active":"2024-04-01T00:13:48.028071+0000","last_peered":"2024-04-01T00:13:48.028071+0000","last_clean":"2024-04-01T00:13:48.028071+0000","last_became_active":"2024-04-01T00:13:47.040849+0000","last_became_peered":"2024-04-01T00:13:47.040849+0000","last_unstale":"2024-04-01T00:13:48.028071+0000","last_undegraded":"2024-04-01T00:13:48.028071+0000","last_fullsized":"2024-04-01T00:13:48.028071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:54.759711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025337+0000","last_change":"2024-04-01T00:13:46.026672+0000","last_active":"2024-04-01T00:13:48.025337+0000","last_peered":"2024-04-01T00:13:48.025337+0000","last_clean":"2024-04-01T00:13:48.025337+0000","last_became_active":"2024-04-01T00:13:46.026365+0000","last_became_peered":"2024-04-01T00:13:46.026365+0000","last_unstale":"2024-04-01T00:13:48.025337+0000","last_undegraded":"2024-04-01T00:13:48.025337+0000","last_fullsized":"2024-04-01T00:13:48.025337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027240+0000","last_change":"2024-04-01T00:13:47.028664+0000","last_active":"2024-04-01T00:13:48.027240+0000","last_peered":"2024-04-01T00:13:48.027240+0000","last_clean":"2024-04-01T00:13:48.027240+0000","last_became_active":"2024-04-01T00:13:47.028387+0000","last_became_peered":"2024-04-01T00:13:47.028387+0000","last_unstale":"2024-04-01T00:13:48.027240+0000","last_undegraded":"2024-04-01T00:13:48.027240+0000","last_fullsized":"2024-04-01T00:13:48.027240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905298+0000","last_change":"2024-04-01T00:13:46.037508+0000","last_active":"2024-04-01T00:13:48.905298+0000","last_peered":"2024-04-01T00:13:48.905298+0000","last_clean":"2024-04-01T00:13:48.905298+0000","last_became_active":"2024-04-01T00:13:46.037083+0000","last_became_peered":"2024-04-01T00:13:46.037083+0000","last_unstale":"2024-04-01T00:13:48.905298+0000","last_undegraded":"2024-04-01T00:13:48.905298+0000","last_fullsized":"2024-04-01T00:13:48.905298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025575+0000","last_change":"2024-04-01T00:13:47.027434+0000","last_active":"2024-04-01T00:13:48.025575+0000","last_peered":"2024-04-01T00:13:48.025575+0000","last_clean":"2024-04-01T00:13:48.025575+0000","last_became_active":"2024-04-01T00:13:47.027134+0000","last_became_peered":"2024-04-01T00:13:47.027134+0000","last_unstale":"2024-04-01T00:13:48.025575+0000","last_undegraded":"2024-04-01T00:13:48.025575+0000","last_fullsized":"2024-04-01T00:13:48.025575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030010+0000","last_change":"2024-04-01T00:13:46.029352+0000","last_active":"2024-04-01T00:13:48.030010+0000","last_peered":"2024-04-01T00:13:48.030010+0000","last_clean":"2024-04-01T00:13:48.030010+0000","last_became_active":"2024-04-01T00:13:46.029053+0000","last_became_peered":"2024-04-01T00:13:46.029053+0000","last_unstale":"2024-04-01T00:13:48.030010+0000","last_undegraded":"2024-04-01T00:13:48.030010+0000","last_fullsized":"2024-04-01T00:13:48.030010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906349+0000","last_change":"2024-04-01T00:13:47.045641+0000","last_active":"2024-04-01T00:13:48.906349+0000","last_peered":"2024-04-01T00:13:48.906349+0000","last_clean":"2024-04-01T00:13:48.906349+0000","last_became_active":"2024-04-01T00:13:47.045348+0000","last_became_peered":"2024-04-01T00:13:47.045348+0000","last_unstale":"2024-04-01T00:13:48.906349+0000","last_undegraded":"2024-04-01T00:13:48.906349+0000","last_fullsized":"2024-04-01T00:13:48.906349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030805+0000","last_change":"2024-04-01T00:13:46.037865+0000","last_active":"2024-04-01T00:13:48.030805+0000","last_peered":"2024-04-01T00:13:48.030805+0000","last_clean":"2024-04-01T00:13:48.030805+0000","last_became_active":"2024-04-01T00:13:46.037564+0000","last_became_peered":"2024-04-01T00:13:46.037564+0000","last_unstale":"2024-04-01T00:13:48.030805+0000","last_undegraded":"2024-04-01T00:13:48.030805+0000","last_fullsized":"2024-04-01T00:13:48.030805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906532+0000","last_change":"2024-04-01T00:13:47.038745+0000","last_active":"2024-04-01T00:13:48.906532+0000","last_peered":"2024-04-01T00:13:48.906532+0000","last_clean":"2024-04-01T00:13:48.906532+0000","last_became_active":"2024-04-01T00:13:47.038440+0000","last_became_peered":"2024-04-01T00:13:47.038440+0000","last_unstale":"2024-04-01T00:13:48.906532+0000","last_undegraded":"2024-04-01T00:13:48.906532+0000","last_fullsized":"2024-04-01T00:13:48.906532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904810+0000","last_change":"2024-04-01T00:13:46.037408+0000","last_active":"2024-04-01T00:13:48.904810+0000","last_peered":"2024-04-01T00:13:48.904810+0000","last_clean":"2024-04-01T00:13:48.904810+0000","last_became_active":"2024-04-01T00:13:46.037120+0000","last_became_peered":"2024-04-01T00:13:46.037120+0000","last_unstale":"2024-04-01T00:13:48.904810+0000","last_undegraded":"2024-04-01T00:13:48.904810+0000","last_fullsized":"2024-04-01T00:13:48.904810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028332+0000","last_change":"2024-04-01T00:13:47.032590+0000","last_active":"2024-04-01T00:13:48.028332+0000","last_peered":"2024-04-01T00:13:48.028332+0000","last_clean":"2024-04-01T00:13:48.028332+0000","last_became_active":"2024-04-01T00:13:47.032294+0000","last_became_peered":"2024-04-01T00:13:47.032294+0000","last_unstale":"2024-04-01T00:13:48.028332+0000","last_undegraded":"2024-04-01T00:13:48.028332+0000","last_fullsized":"2024-04-01T00:13:48.028332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025536+0000","last_change":"2024-04-01T00:13:46.036386+0000","last_active":"2024-04-01T00:13:48.025536+0000","last_peered":"2024-04-01T00:13:48.025536+0000","last_clean":"2024-04-01T00:13:48.025536+0000","last_became_active":"2024-04-01T00:13:46.036055+0000","last_became_peered":"2024-04-01T00:13:46.036055+0000","last_unstale":"2024-04-01T00:13:48.025536+0000","last_undegraded":"2024-04-01T00:13:48.025536+0000","last_fullsized":"2024-04-01T00:13:48.025536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905628+0000","last_change":"2024-04-01T00:13:47.027936+0000","last_active":"2024-04-01T00:13:48.905628+0000","last_peered":"2024-04-01T00:13:48.905628+0000","last_clean":"2024-04-01T00:13:48.905628+0000","last_became_active":"2024-04-01T00:13:47.027628+0000","last_became_peered":"2024-04-01T00:13:47.027628+0000","last_unstale":"2024-04-01T00:13:48.905628+0000","last_undegraded":"2024-04-01T00:13:48.905628+0000","last_fullsized":"2024-04-01T00:13:48.905628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025431+0000","last_change":"2024-04-01T00:13:46.028586+0000","last_active":"2024-04-01T00:13:48.025431+0000","last_peered":"2024-04-01T00:13:48.025431+0000","last_clean":"2024-04-01T00:13:48.025431+0000","last_became_active":"2024-04-01T00:13:46.028309+0000","last_became_peered":"2024-04-01T00:13:46.028309+0000","last_unstale":"2024-04-01T00:13:48.025431+0000","last_undegraded":"2024-04-01T00:13:48.025431+0000","last_fullsized":"2024-04-01T00:13:48.025431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906125+0000","last_change":"2024-04-01T00:13:47.042379+0000","last_active":"2024-04-01T00:13:48.906125+0000","last_peered":"2024-04-01T00:13:48.906125+0000","last_clean":"2024-04-01T00:13:48.906125+0000","last_became_active":"2024-04-01T00:13:47.042018+0000","last_became_peered":"2024-04-01T00:13:47.042018+0000","last_unstale":"2024-04-01T00:13:48.906125+0000","last_undegraded":"2024-04-01T00:13:48.906125+0000","last_fullsized":"2024-04-01T00:13:48.906125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905276+0000","last_change":"2024-04-01T00:13:46.037179+0000","last_active":"2024-04-01T00:13:48.905276+0000","last_peered":"2024-04-01T00:13:48.905276+0000","last_clean":"2024-04-01T00:13:48.905276+0000","last_became_active":"2024-04-01T00:13:46.036168+0000","last_became_peered":"2024-04-01T00:13:46.036168+0000","last_unstale":"2024-04-01T00:13:48.905276+0000","last_undegraded":"2024-04-01T00:13:48.905276+0000","last_fullsized":"2024-04-01T00:13:48.905276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030533+0000","last_change":"2024-04-01T00:13:47.031588+0000","last_active":"2024-04-01T00:13:48.030533+0000","last_peered":"2024-04-01T00:13:48.030533+0000","last_clean":"2024-04-01T00:13:48.030533+0000","last_became_active":"2024-04-01T00:13:47.031297+0000","last_became_peered":"2024-04-01T00:13:47.031297+0000","last_unstale":"2024-04-01T00:13:48.030533+0000","last_undegraded":"2024-04-01T00:13:48.030533+0000","last_fullsized":"2024-04-01T00:13:48.030533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027184+0000","last_change":"2024-04-01T00:13:47.036405+0000","last_active":"2024-04-01T00:13:48.027184+0000","last_peered":"2024-04-01T00:13:48.027184+0000","last_clean":"2024-04-01T00:13:48.027184+0000","last_became_active":"2024-04-01T00:13:47.036144+0000","last_became_peered":"2024-04-01T00:13:47.036144+0000","last_unstale":"2024-04-01T00:13:48.027184+0000","last_undegraded":"2024-04-01T00:13:48.027184+0000","last_fullsized":"2024-04-01T00:13:48.027184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905062+0000","last_change":"2024-04-01T00:13:46.036339+0000","last_active":"2024-04-01T00:13:48.905062+0000","last_peered":"2024-04-01T00:13:48.905062+0000","last_clean":"2024-04-01T00:13:48.905062+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.905062+0000","last_undegraded":"2024-04-01T00:13:48.905062+0000","last_fullsized":"2024-04-01T00:13:48.905062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:50.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026918+0000","last_change":"2024-04-01T00:13:47.037119+0000","last_active":"2024-04-01T00:13:48.026918+0000","last_peered":"2024-04-01T00:13:48.026918+0000","last_clean":"2024-04-01T00:13:48.026918+0000","last_became_active":"2024-04-01T00:13:47.036860+0000","last_became_peered":"2024-04-01T00:13:47.036860+0000","last_unstale":"2024-04-01T00:13:48.026918+0000","last_undegraded":"2024-04-01T00:13:48.026918+0000","last_fullsized":"2024-04-01T00:13:48.026918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905203+0000","last_change":"2024-04-01T00:13:46.036988+0000","last_active":"2024-04-01T00:13:48.905203+0000","last_peered":"2024-04-01T00:13:48.905203+0000","last_clean":"2024-04-01T00:13:48.905203+0000","last_became_active":"2024-04-01T00:13:46.036696+0000","last_became_peered":"2024-04-01T00:13:46.036696+0000","last_unstale":"2024-04-01T00:13:48.905203+0000","last_undegraded":"2024-04-01T00:13:48.905203+0000","last_fullsized":"2024-04-01T00:13:48.905203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906528+0000","last_change":"2024-04-01T00:13:47.041150+0000","last_active":"2024-04-01T00:13:48.906528+0000","last_peered":"2024-04-01T00:13:48.906528+0000","last_clean":"2024-04-01T00:13:48.906528+0000","last_became_active":"2024-04-01T00:13:47.040826+0000","last_became_peered":"2024-04-01T00:13:47.040826+0000","last_unstale":"2024-04-01T00:13:48.906528+0000","last_undegraded":"2024-04-01T00:13:48.906528+0000","last_fullsized":"2024-04-01T00:13:48.906528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906154+0000","last_change":"2024-04-01T00:13:46.032149+0000","last_active":"2024-04-01T00:13:48.906154+0000","last_peered":"2024-04-01T00:13:48.906154+0000","last_clean":"2024-04-01T00:13:48.906154+0000","last_became_active":"2024-04-01T00:13:46.031821+0000","last_became_peered":"2024-04-01T00:13:46.031821+0000","last_unstale":"2024-04-01T00:13:48.906154+0000","last_undegraded":"2024-04-01T00:13:48.906154+0000","last_fullsized":"2024-04-01T00:13:48.906154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905922+0000","last_change":"2024-04-01T00:13:47.031502+0000","last_active":"2024-04-01T00:13:48.905922+0000","last_peered":"2024-04-01T00:13:48.905922+0000","last_clean":"2024-04-01T00:13:48.905922+0000","last_became_active":"2024-04-01T00:13:47.030468+0000","last_became_peered":"2024-04-01T00:13:47.030468+0000","last_unstale":"2024-04-01T00:13:48.905922+0000","last_undegraded":"2024-04-01T00:13:48.905922+0000","last_fullsized":"2024-04-01T00:13:48.905922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906670+0000","last_change":"2024-04-01T00:13:46.038424+0000","last_active":"2024-04-01T00:13:48.906670+0000","last_peered":"2024-04-01T00:13:48.906670+0000","last_clean":"2024-04-01T00:13:48.906670+0000","last_became_active":"2024-04-01T00:13:46.038079+0000","last_became_peered":"2024-04-01T00:13:46.038079+0000","last_unstale":"2024-04-01T00:13:48.906670+0000","last_undegraded":"2024-04-01T00:13:48.906670+0000","last_fullsized":"2024-04-01T00:13:48.906670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:44:48.269461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:51.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-01T00:13:51.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.172+0000 7fc54078a640 1 Processor -- start 2024-04-01T00:13:51.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.172+0000 7fc54078a640 1 -- start start 2024-04-01T00:13:51.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc54078a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5380d1920 0x7fc5380d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc54078a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc53806c3e0 0x7fc5380d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc54078a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380d2780 con 0x7fc53815e3f0 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc54078a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380cf3a0 con 0x7fc53806c3e0 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc54078a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380d2240 con 0x7fc5380d1920 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc53806c3e0 0x7fc5380d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc53806c3e0 0x7fc5380d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53418/0 (socket says 172.21.15.64:53418) 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53dd25640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc53815e3f0 0x7fc538161ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42950/0 (socket says 172.21.15.64:42950) 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 -- 172.21.15.64:0/1484369242 learned_addr learned my addr 172.21.15.64:0/1484369242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53e526640 1 --2- 172.21.15.64:0/1484369242 >> v2:172.21.15.100:3300/0 conn(0x7fc5380d1920 0x7fc5380d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 -- 172.21.15.64:0/1484369242 >> v2:172.21.15.100:3300/0 conn(0x7fc5380d1920 msgr2=0x7fc5380d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 --2- 172.21.15.64:0/1484369242 >> v2:172.21.15.100:3300/0 conn(0x7fc5380d1920 0x7fc5380d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 -- 172.21.15.64:0/1484369242 >> v1:172.21.15.64:6789/0 conn(0x7fc53815e3f0 legacy=0x7fc538161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53ed27640 1 -- 172.21.15.64:0/1484369242 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380ce790 con 0x7fc53806c3e0 2024-04-01T00:13:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.173+0000 7fc53d524640 1 -- 172.21.15.64:0/1484369242 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623811564 0 0) 0x7fc5380d2780 con 0x7fc53815e3f0 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.174+0000 7fc53ed27640 1 --2- 172.21.15.64:0/1484369242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53806c3e0 0x7fc5380d0f10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5380b4930 tx=0x7fc53002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525f275d00a752f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.174+0000 7fc53d524640 1 -- 172.21.15.64:0/1484369242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530034070 con 0x7fc53806c3e0 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.174+0000 7fc53d524640 1 -- 172.21.15.64:0/1484369242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc530030860 con 0x7fc53806c3e0 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.174+0000 7fc53d524640 1 -- 172.21.15.64:0/1484369242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530030c30 con 0x7fc53806c3e0 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 -- 172.21.15.64:0/1484369242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53806c3e0 msgr2=0x7fc5380d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 --2- 172.21.15.64:0/1484369242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53806c3e0 0x7fc5380d0f10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5380b4930 tx=0x7fc53002b9a0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 -- 172.21.15.64:0/1484369242 shutdown_connections 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 --2- 172.21.15.64:0/1484369242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53806c3e0 0x7fc5380d0f10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 --2- 172.21.15.64:0/1484369242 >> v2:172.21.15.100:3300/0 conn(0x7fc5380d1920 0x7fc5380d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 -- 172.21.15.64:0/1484369242 >> 172.21.15.64:0/1484369242 conn(0x7fc53809a980 msgr2=0x7fc53809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 -- 172.21.15.64:0/1484369242 shutdown_connections 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.175+0000 7fc54078a640 1 -- 172.21.15.64:0/1484369242 wait complete. 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.176+0000 7fc54078a640 1 Processor -- start 2024-04-01T00:13:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.176+0000 7fc54078a640 1 -- start start 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53806c3e0 0x7fc538156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5380b2160 0x7fc5380b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380cf030 con 0x7fc5380d1920 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380ced50 con 0x7fc53806c3e0 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc5380b0cd0 con 0x7fc5380b2160 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53430/0 (socket says 172.21.15.64:53430) 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 -- 172.21.15.64:0/3440702436 learned_addr learned my addr 172.21.15.64:0/3440702436 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53e526640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53806c3e0 0x7fc538156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53ed27640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5380b2160 0x7fc5380b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 -- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5380b2160 msgr2=0x7fc5380b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5380b2160 0x7fc5380b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 -- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53806c3e0 msgr2=0x7fc538156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53806c3e0 0x7fc538156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 -- 172.21.15.64:0/3440702436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381e8210 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc53dd25640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c420 tx=0x7fc52c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0074e0 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52c0047e0 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5380b6ee0 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.177+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004c10 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.178+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5380b6920 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.178+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5380681a0 con 0x7fc5380d1920 2024-04-01T00:13:51.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.179+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fc52c011070 con 0x7fc5380d1920 2024-04-01T00:13:51.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.179+0000 7fc536ffd640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 0x7fc52406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.179+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc52c08a350 con 0x7fc5380d1920 2024-04-01T00:13:51.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.179+0000 7fc53e526640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 0x7fc52406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.180+0000 7fc53e526640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 0x7fc52406de50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5381e9bd0 tx=0x7fc52800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.183+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fc52c058910 con 0x7fc5380d1920 2024-04-01T00:13:51.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.433+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc53806f050 con 0x7fc5380d1920 2024-04-01T00:13:51.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.433+0000 7fc536ffd640 1 -- 172.21.15.64:0/3440702436 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7fc52c05c9b0 con 0x7fc5380d1920 2024-04-01T00:13:51.434 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:51.434 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1711930413,"num_in_osds":8,"osd_in_since":1711930404,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224251904,"bytes_avail":772869861376,"bytes_total":773094113280,"write_bytes_sec":2497,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4272}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-01T00:13:22.527884+0000","services":{}},"progress_events":{}} 2024-04-01T00:13:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 msgr2=0x7fc52406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 0x7fc52406de50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5381e9bd0 tx=0x7fc52800b040 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 msgr2=0x7fc5380b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c420 tx=0x7fc52c00c8e0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 shutdown_connections 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc52406b990 0x7fc52406de50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5380b2160 0x7fc5380b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5380d1920 0x7fc5380b1c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 --2- 172.21.15.64:0/3440702436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53806c3e0 0x7fc538156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 >> 172.21.15.64:0/3440702436 conn(0x7fc53809a980 msgr2=0x7fc53806f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 shutdown_connections 2024-04-01T00:13:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.435+0000 7fc54078a640 1 -- 172.21.15.64:0/3440702436 wait complete. 2024-04-01T00:13:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-01T00:13:51.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:51.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.548+0000 7f07d209d640 1 Processor -- start 2024-04-01T00:13:51.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.548+0000 7f07d209d640 1 -- start start 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07d209d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07cc15e1c0 0x7f07cc0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07d209d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07cc0cc340 0x7f07cc0d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07d209d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07cc0d0060 con 0x7f07cc06c390 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07d209d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f07cc0d0650 con 0x7f07cc15e1c0 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07d209d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f07cc06d100 con 0x7f07cc0cc340 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07cc15e1c0 0x7f07cc0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07cc15e1c0 0x7f07cc0d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53432/0 (socket says 172.21.15.64:53432) 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 -- 172.21.15.64:0/1266182028 learned_addr learned my addr 172.21.15.64:0/1266182028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07caffd640 1 --2- 172.21.15.64:0/1266182028 >> v2:172.21.15.100:3300/0 conn(0x7f07cc0cc340 0x7f07cc0d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 -- 172.21.15.64:0/1266182028 >> v2:172.21.15.100:3300/0 conn(0x7f07cc0cc340 msgr2=0x7f07cc0d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 --2- 172.21.15.64:0/1266182028 >> v2:172.21.15.100:3300/0 conn(0x7f07cc0cc340 0x7f07cc0d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 -- 172.21.15.64:0/1266182028 >> v1:172.21.15.100:6789/0 conn(0x7f07cc06c390 legacy=0x7f07cc06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.549+0000 7f07cb7fe640 1 -- 172.21.15.64:0/1266182028 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc06d3e0 con 0x7f07cc15e1c0 2024-04-01T00:13:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07cb7fe640 1 --2- 172.21.15.64:0/1266182028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc15e1c0 0x7f07cc0d1c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07cc072e50 tx=0x7f07bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250eec88595aa09f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07ca7fc640 1 -- 172.21.15.64:0/1266182028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc035020 con 0x7f07cc15e1c0 2024-04-01T00:13:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07ca7fc640 1 -- 172.21.15.64:0/1266182028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07bc031870 con 0x7f07cc15e1c0 2024-04-01T00:13:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07ca7fc640 1 -- 172.21.15.64:0/1266182028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc031c40 con 0x7f07cc15e1c0 2024-04-01T00:13:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07d209d640 1 -- 172.21.15.64:0/1266182028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc15e1c0 msgr2=0x7f07cc0d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07d209d640 1 --2- 172.21.15.64:0/1266182028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc15e1c0 0x7f07cc0d1c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07cc072e50 tx=0x7f07bc02b910 comp rx=0 tx=0).stop 2024-04-01T00:13:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07d209d640 1 -- 172.21.15.64:0/1266182028 shutdown_connections 2024-04-01T00:13:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07d209d640 1 --2- 172.21.15.64:0/1266182028 >> v2:172.21.15.100:3300/0 conn(0x7f07cc0cc340 0x7f07cc0d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.550+0000 7f07d209d640 1 --2- 172.21.15.64:0/1266182028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc15e1c0 0x7f07cc0d1c10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.551+0000 7f07d209d640 1 -- 172.21.15.64:0/1266182028 >> 172.21.15.64:0/1266182028 conn(0x7f07cc09a980 msgr2=0x7f07cc09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.551+0000 7f07d209d640 1 -- 172.21.15.64:0/1266182028 shutdown_connections 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.551+0000 7f07d209d640 1 -- 172.21.15.64:0/1266182028 wait complete. 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.552+0000 7f07d209d640 1 Processor -- start 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.552+0000 7f07d209d640 1 -- start start 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc0cc340 0x7f07cc15f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15e1c0 0x7f07cc0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0b0b80 con 0x7f07cc06c390 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0b0a00 con 0x7f07cc0cc340 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc0b0d00 con 0x7f07cc15e1c0 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53440/0 (socket says 172.21.15.64:53440) 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 -- 172.21.15.64:0/2261221737 learned_addr learned my addr 172.21.15.64:0/2261221737 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07caffd640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc0cc340 0x7f07cc15f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cbfff640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15e1c0 0x7f07cc0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 -- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15e1c0 msgr2=0x7f07cc0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15e1c0 0x7f07cc0afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 -- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc0cc340 msgr2=0x7f07cc15f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc0cc340 0x7f07cc15f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 -- 172.21.15.64:0/2261221737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc160170 con 0x7f07cc06c390 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07cb7fe640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07bc000c00 tx=0x7f07bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc035040 con 0x7f07cc06c390 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07bc030d00 con 0x7f07cc06c390 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0456a0 con 0x7f07cc06c390 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc160450 con 0x7f07cc06c390 2024-04-01T00:13:51.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.553+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc160a10 con 0x7f07cc06c390 2024-04-01T00:13:51.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.554+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc0681a0 con 0x7f07cc06c390 2024-04-01T00:13:51.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.555+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f07bc038020 con 0x7f07cc06c390 2024-04-01T00:13:51.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.555+0000 7f07b3fff640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 0x7f07b806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:51.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.555+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07bc0ac9a0 con 0x7f07cc06c390 2024-04-01T00:13:51.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.555+0000 7f07caffd640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 0x7f07b806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:51.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.556+0000 7f07caffd640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 0x7f07b806de50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07cc1f7140 tx=0x7f07c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:51.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.559+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f07bc07aff0 con 0x7f07cc06c390 2024-04-01T00:13:51.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.738+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07cc06cb00 con 0x7f07b806b990 2024-04-01T00:13:51.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.745+0000 7f07b3fff640 1 -- 172.21.15.64:0/2261221737 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7f07cc06cb00 con 0x7f07b806b990 2024-04-01T00:13:51.748 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:51.750 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:51.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 msgr2=0x7f07b806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 0x7f07b806de50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07cc1f7140 tx=0x7f07c0009190 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 msgr2=0x7f07cc1f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07bc000c00 tx=0x7f07bc0046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 shutdown_connections 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07b806b990 0x7f07b806de50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15e1c0 0x7f07cc0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc0cc340 0x7f07cc15f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 --2- 172.21.15.64:0/2261221737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07cc06c390 0x7f07cc1f0ff0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 >> 172.21.15.64:0/2261221737 conn(0x7f07cc09a980 msgr2=0x7f07cc0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 shutdown_connections 2024-04-01T00:13:51.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:51.748+0000 7f07d209d640 1 -- 172.21.15.64:0/2261221737 wait complete. 2024-04-01T00:13:51.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-01T00:13:50.746916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.739501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905422+0000","last_change":"2024-04-01T00:13:47.045277+0000","last_active":"2024-04-01T00:13:48.905422+0000","last_peered":"2024-04-01T00:13:48.905422+0000","last_clean":"2024-04-01T00:13:48.905422+0000","last_became_active":"2024-04-01T00:13:47.044728+0000","last_became_peered":"2024-04-01T00:13:47.044728+0000","last_unstale":"2024-04-01T00:13:48.905422+0000","last_undegraded":"2024-04-01T00:13:48.905422+0000","last_fullsized":"2024-04-01T00:13:48.905422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:50:54.919900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025542+0000","last_change":"2024-04-01T00:13:47.039324+0000","last_active":"2024-04-01T00:13:48.025542+0000","last_peered":"2024-04-01T00:13:48.025542+0000","last_clean":"2024-04-01T00:13:48.025542+0000","last_became_active":"2024-04-01T00:13:47.039002+0000","last_became_peered":"2024-04-01T00:13:47.039002+0000","last_unstale":"2024-04-01T00:13:48.025542+0000","last_undegraded":"2024-04-01T00:13:48.025542+0000","last_fullsized":"2024-04-01T00:13:48.025542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906198+0000","last_change":"2024-04-01T00:13:47.039670+0000","last_active":"2024-04-01T00:13:48.906198+0000","last_peered":"2024-04-01T00:13:48.906198+0000","last_clean":"2024-04-01T00:13:48.906198+0000","last_became_active":"2024-04-01T00:13:47.037096+0000","last_became_peered":"2024-04-01T00:13:47.037096+0000","last_unstale":"2024-04-01T00:13:48.906198+0000","last_undegraded":"2024-04-01T00:13:48.906198+0000","last_fullsized":"2024-04-01T00:13:48.906198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:05:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026858+0000","last_change":"2024-04-01T00:13:47.033997+0000","last_active":"2024-04-01T00:13:48.026858+0000","last_peered":"2024-04-01T00:13:48.026858+0000","last_clean":"2024-04-01T00:13:48.026858+0000","last_became_active":"2024-04-01T00:13:47.033682+0000","last_became_peered":"2024-04-01T00:13:47.033682+0000","last_unstale":"2024-04-01T00:13:48.026858+0000","last_undegraded":"2024-04-01T00:13:48.026858+0000","last_fullsized":"2024-04-01T00:13:48.026858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905169+0000","last_change":"2024-04-01T00:13:47.040873+0000","last_active":"2024-04-01T00:13:48.905169+0000","last_peered":"2024-04-01T00:13:48.905169+0000","last_clean":"2024-04-01T00:13:48.905169+0000","last_became_active":"2024-04-01T00:13:47.039953+0000","last_became_peered":"2024-04-01T00:13:47.039953+0000","last_unstale":"2024-04-01T00:13:48.905169+0000","last_undegraded":"2024-04-01T00:13:48.905169+0000","last_fullsized":"2024-04-01T00:13:48.905169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026638+0000","last_change":"2024-04-01T00:13:47.041029+0000","last_active":"2024-04-01T00:13:48.026638+0000","last_peered":"2024-04-01T00:13:48.026638+0000","last_clean":"2024-04-01T00:13:48.026638+0000","last_became_active":"2024-04-01T00:13:47.040771+0000","last_became_peered":"2024-04-01T00:13:47.040771+0000","last_unstale":"2024-04-01T00:13:48.026638+0000","last_undegraded":"2024-04-01T00:13:48.026638+0000","last_fullsized":"2024-04-01T00:13:48.026638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904941+0000","last_change":"2024-04-01T00:13:47.042649+0000","last_active":"2024-04-01T00:13:48.904941+0000","last_peered":"2024-04-01T00:13:48.904941+0000","last_clean":"2024-04-01T00:13:48.904941+0000","last_became_active":"2024-04-01T00:13:47.042373+0000","last_became_peered":"2024-04-01T00:13:47.042373+0000","last_unstale":"2024-04-01T00:13:48.904941+0000","last_undegraded":"2024-04-01T00:13:48.904941+0000","last_fullsized":"2024-04-01T00:13:48.904941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905488+0000","last_change":"2024-04-01T00:13:47.031258+0000","last_active":"2024-04-01T00:13:48.905488+0000","last_peered":"2024-04-01T00:13:48.905488+0000","last_clean":"2024-04-01T00:13:48.905488+0000","last_became_active":"2024-04-01T00:13:47.030972+0000","last_became_peered":"2024-04-01T00:13:47.030972+0000","last_unstale":"2024-04-01T00:13:48.905488+0000","last_undegraded":"2024-04-01T00:13:48.905488+0000","last_fullsized":"2024-04-01T00:13:48.905488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905452+0000","last_change":"2024-04-01T00:13:47.045417+0000","last_active":"2024-04-01T00:13:48.905452+0000","last_peered":"2024-04-01T00:13:48.905452+0000","last_clean":"2024-04-01T00:13:48.905452+0000","last_became_active":"2024-04-01T00:13:47.044978+0000","last_became_peered":"2024-04-01T00:13:47.044978+0000","last_unstale":"2024-04-01T00:13:48.905452+0000","last_undegraded":"2024-04-01T00:13:48.905452+0000","last_fullsized":"2024-04-01T00:13:48.905452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:12:06.404460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030825+0000","last_change":"2024-04-01T00:13:47.035602+0000","last_active":"2024-04-01T00:13:48.030825+0000","last_peered":"2024-04-01T00:13:48.030825+0000","last_clean":"2024-04-01T00:13:48.030825+0000","last_became_active":"2024-04-01T00:13:47.035293+0000","last_became_peered":"2024-04-01T00:13:47.035293+0000","last_unstale":"2024-04-01T00:13:48.030825+0000","last_undegraded":"2024-04-01T00:13:48.030825+0000","last_fullsized":"2024-04-01T00:13:48.030825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030712+0000","last_change":"2024-04-01T00:13:47.037792+0000","last_active":"2024-04-01T00:13:48.030712+0000","last_peered":"2024-04-01T00:13:48.030712+0000","last_clean":"2024-04-01T00:13:48.030712+0000","last_became_active":"2024-04-01T00:13:47.037508+0000","last_became_peered":"2024-04-01T00:13:47.037508+0000","last_unstale":"2024-04-01T00:13:48.030712+0000","last_undegraded":"2024-04-01T00:13:48.030712+0000","last_fullsized":"2024-04-01T00:13:48.030712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028123+0000","last_change":"2024-04-01T00:13:47.026838+0000","last_active":"2024-04-01T00:13:48.028123+0000","last_peered":"2024-04-01T00:13:48.028123+0000","last_clean":"2024-04-01T00:13:48.028123+0000","last_became_active":"2024-04-01T00:13:47.026586+0000","last_became_peered":"2024-04-01T00:13:47.026586+0000","last_unstale":"2024-04-01T00:13:48.028123+0000","last_undegraded":"2024-04-01T00:13:48.028123+0000","last_fullsized":"2024-04-01T00:13:48.028123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:13.513798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905964+0000","last_change":"2024-04-01T00:13:47.028848+0000","last_active":"2024-04-01T00:13:48.905964+0000","last_peered":"2024-04-01T00:13:48.905964+0000","last_clean":"2024-04-01T00:13:48.905964+0000","last_became_active":"2024-04-01T00:13:47.028577+0000","last_became_peered":"2024-04-01T00:13:47.028577+0000","last_unstale":"2024-04-01T00:13:48.905964+0000","last_undegraded":"2024-04-01T00:13:48.905964+0000","last_fullsized":"2024-04-01T00:13:48.905964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906018+0000","last_change":"2024-04-01T00:13:47.039847+0000","last_active":"2024-04-01T00:13:48.906018+0000","last_peered":"2024-04-01T00:13:48.906018+0000","last_clean":"2024-04-01T00:13:48.906018+0000","last_became_active":"2024-04-01T00:13:47.039459+0000","last_became_peered":"2024-04-01T00:13:47.039459+0000","last_unstale":"2024-04-01T00:13:48.906018+0000","last_undegraded":"2024-04-01T00:13:48.906018+0000","last_fullsized":"2024-04-01T00:13:48.906018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:51.654019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025413+0000","last_change":"2024-04-01T00:13:47.034288+0000","last_active":"2024-04-01T00:13:48.025413+0000","last_peered":"2024-04-01T00:13:48.025413+0000","last_clean":"2024-04-01T00:13:48.025413+0000","last_became_active":"2024-04-01T00:13:47.034022+0000","last_became_peered":"2024-04-01T00:13:47.034022+0000","last_unstale":"2024-04-01T00:13:48.025413+0000","last_undegraded":"2024-04-01T00:13:48.025413+0000","last_fullsized":"2024-04-01T00:13:48.025413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026557+0000","last_change":"2024-04-01T00:13:47.039173+0000","last_active":"2024-04-01T00:13:48.026557+0000","last_peered":"2024-04-01T00:13:48.026557+0000","last_clean":"2024-04-01T00:13:48.026557+0000","last_became_active":"2024-04-01T00:13:47.038845+0000","last_became_peered":"2024-04-01T00:13:47.038845+0000","last_unstale":"2024-04-01T00:13:48.026557+0000","last_undegraded":"2024-04-01T00:13:48.026557+0000","last_fullsized":"2024-04-01T00:13:48.026557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027411+0000","last_change":"2024-04-01T00:13:47.043400+0000","last_active":"2024-04-01T00:13:48.027411+0000","last_peered":"2024-04-01T00:13:48.027411+0000","last_clean":"2024-04-01T00:13:48.027411+0000","last_became_active":"2024-04-01T00:13:47.043120+0000","last_became_peered":"2024-04-01T00:13:47.043120+0000","last_unstale":"2024-04-01T00:13:48.027411+0000","last_undegraded":"2024-04-01T00:13:48.027411+0000","last_fullsized":"2024-04-01T00:13:48.027411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905429+0000","last_change":"2024-04-01T00:13:47.035436+0000","last_active":"2024-04-01T00:13:48.905429+0000","last_peered":"2024-04-01T00:13:48.905429+0000","last_clean":"2024-04-01T00:13:48.905429+0000","last_became_active":"2024-04-01T00:13:47.035072+0000","last_became_peered":"2024-04-01T00:13:47.035072+0000","last_unstale":"2024-04-01T00:13:48.905429+0000","last_undegraded":"2024-04-01T00:13:48.905429+0000","last_fullsized":"2024-04-01T00:13:48.905429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905379+0000","last_change":"2024-04-01T00:13:47.040506+0000","last_active":"2024-04-01T00:13:48.905379+0000","last_peered":"2024-04-01T00:13:48.905379+0000","last_clean":"2024-04-01T00:13:48.905379+0000","last_became_active":"2024-04-01T00:13:47.040244+0000","last_became_peered":"2024-04-01T00:13:47.040244+0000","last_unstale":"2024-04-01T00:13:48.905379+0000","last_undegraded":"2024-04-01T00:13:48.905379+0000","last_fullsized":"2024-04-01T00:13:48.905379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030707+0000","last_change":"2024-04-01T00:13:47.031532+0000","last_active":"2024-04-01T00:13:48.030707+0000","last_peered":"2024-04-01T00:13:48.030707+0000","last_clean":"2024-04-01T00:13:48.030707+0000","last_became_active":"2024-04-01T00:13:47.031252+0000","last_became_peered":"2024-04-01T00:13:47.031252+0000","last_unstale":"2024-04-01T00:13:48.030707+0000","last_undegraded":"2024-04-01T00:13:48.030707+0000","last_fullsized":"2024-04-01T00:13:48.030707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906466+0000","last_change":"2024-04-01T00:13:47.047404+0000","last_active":"2024-04-01T00:13:48.906466+0000","last_peered":"2024-04-01T00:13:48.906466+0000","last_clean":"2024-04-01T00:13:48.906466+0000","last_became_active":"2024-04-01T00:13:47.047080+0000","last_became_peered":"2024-04-01T00:13:47.047080+0000","last_unstale":"2024-04-01T00:13:48.906466+0000","last_undegraded":"2024-04-01T00:13:48.906466+0000","last_fullsized":"2024-04-01T00:13:48.906466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026698+0000","last_change":"2024-04-01T00:13:47.033044+0000","last_active":"2024-04-01T00:13:48.026698+0000","last_peered":"2024-04-01T00:13:48.026698+0000","last_clean":"2024-04-01T00:13:48.026698+0000","last_became_active":"2024-04-01T00:13:47.032783+0000","last_became_peered":"2024-04-01T00:13:47.032783+0000","last_unstale":"2024-04-01T00:13:48.026698+0000","last_undegraded":"2024-04-01T00:13:48.026698+0000","last_fullsized":"2024-04-01T00:13:48.026698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905485+0000","last_change":"2024-04-01T00:13:47.035282+0000","last_active":"2024-04-01T00:13:48.905485+0000","last_peered":"2024-04-01T00:13:48.905485+0000","last_clean":"2024-04-01T00:13:48.905485+0000","last_became_active":"2024-04-01T00:13:47.034953+0000","last_became_peered":"2024-04-01T00:13:47.034953+0000","last_unstale":"2024-04-01T00:13:48.905485+0000","last_undegraded":"2024-04-01T00:13:48.905485+0000","last_fullsized":"2024-04-01T00:13:48.905485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026661+0000","last_change":"2024-04-01T00:13:47.034177+0000","last_active":"2024-04-01T00:13:48.026661+0000","last_peered":"2024-04-01T00:13:48.026661+0000","last_clean":"2024-04-01T00:13:48.026661+0000","last_became_active":"2024-04-01T00:13:47.033426+0000","last_became_peered":"2024-04-01T00:13:47.033426+0000","last_unstale":"2024-04-01T00:13:48.026661+0000","last_undegraded":"2024-04-01T00:13:48.026661+0000","last_fullsized":"2024-04-01T00:13:48.026661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906196+0000","last_change":"2024-04-01T00:13:47.034202+0000","last_active":"2024-04-01T00:13:48.906196+0000","last_peered":"2024-04-01T00:13:48.906196+0000","last_clean":"2024-04-01T00:13:48.906196+0000","last_became_active":"2024-04-01T00:13:47.033941+0000","last_became_peered":"2024-04-01T00:13:47.033941+0000","last_unstale":"2024-04-01T00:13:48.906196+0000","last_undegraded":"2024-04-01T00:13:48.906196+0000","last_fullsized":"2024-04-01T00:13:48.906196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030886+0000","last_change":"2024-04-01T00:13:47.042844+0000","last_active":"2024-04-01T00:13:48.030886+0000","last_peered":"2024-04-01T00:13:48.030886+0000","last_clean":"2024-04-01T00:13:48.030886+0000","last_became_active":"2024-04-01T00:13:47.042518+0000","last_became_peered":"2024-04-01T00:13:47.042518+0000","last_unstale":"2024-04-01T00:13:48.030886+0000","last_undegraded":"2024-04-01T00:13:48.030886+0000","last_fullsized":"2024-04-01T00:13:48.030886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025478+0000","last_change":"2024-04-01T00:13:47.032968+0000","last_active":"2024-04-01T00:13:48.025478+0000","last_peered":"2024-04-01T00:13:48.025478+0000","last_clean":"2024-04-01T00:13:48.025478+0000","last_became_active":"2024-04-01T00:13:47.032695+0000","last_became_peered":"2024-04-01T00:13:47.032695+0000","last_unstale":"2024-04-01T00:13:48.025478+0000","last_undegraded":"2024-04-01T00:13:48.025478+0000","last_fullsized":"2024-04-01T00:13:48.025478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906706+0000","last_change":"2024-04-01T00:13:47.035447+0000","last_active":"2024-04-01T00:13:48.906706+0000","last_peered":"2024-04-01T00:13:48.906706+0000","last_clean":"2024-04-01T00:13:48.906706+0000","last_became_active":"2024-04-01T00:13:47.035003+0000","last_became_peered":"2024-04-01T00:13:47.035003+0000","last_unstale":"2024-04-01T00:13:48.906706+0000","last_undegraded":"2024-04-01T00:13:48.906706+0000","last_fullsized":"2024-04-01T00:13:48.906706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906541+0000","last_change":"2024-04-01T00:13:47.039218+0000","last_active":"2024-04-01T00:13:48.906541+0000","last_peered":"2024-04-01T00:13:48.906541+0000","last_clean":"2024-04-01T00:13:48.906541+0000","last_became_active":"2024-04-01T00:13:47.037984+0000","last_became_peered":"2024-04-01T00:13:47.037984+0000","last_unstale":"2024-04-01T00:13:48.906541+0000","last_undegraded":"2024-04-01T00:13:48.906541+0000","last_fullsized":"2024-04-01T00:13:48.906541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026792+0000","last_change":"2024-04-01T00:13:47.034521+0000","last_active":"2024-04-01T00:13:48.026792+0000","last_peered":"2024-04-01T00:13:48.026792+0000","last_clean":"2024-04-01T00:13:48.026792+0000","last_became_active":"2024-04-01T00:13:47.034251+0000","last_became_peered":"2024-04-01T00:13:47.034251+0000","last_unstale":"2024-04-01T00:13:48.026792+0000","last_undegraded":"2024-04-01T00:13:48.026792+0000","last_fullsized":"2024-04-01T00:13:48.026792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905386+0000","last_change":"2024-04-01T00:13:47.035817+0000","last_active":"2024-04-01T00:13:48.905386+0000","last_peered":"2024-04-01T00:13:48.905386+0000","last_clean":"2024-04-01T00:13:48.905386+0000","last_became_active":"2024-04-01T00:13:47.034834+0000","last_became_peered":"2024-04-01T00:13:47.034834+0000","last_unstale":"2024-04-01T00:13:48.905386+0000","last_undegraded":"2024-04-01T00:13:48.905386+0000","last_fullsized":"2024-04-01T00:13:48.905386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030497+0000","last_change":"2024-04-01T00:13:47.039829+0000","last_active":"2024-04-01T00:13:48.030497+0000","last_peered":"2024-04-01T00:13:48.030497+0000","last_clean":"2024-04-01T00:13:48.030497+0000","last_became_active":"2024-04-01T00:13:47.039517+0000","last_became_peered":"2024-04-01T00:13:47.039517+0000","last_unstale":"2024-04-01T00:13:48.030497+0000","last_undegraded":"2024-04-01T00:13:48.030497+0000","last_fullsized":"2024-04-01T00:13:48.030497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027883+0000","last_change":"2024-04-01T00:13:47.026124+0000","last_active":"2024-04-01T00:13:48.027883+0000","last_peered":"2024-04-01T00:13:48.027883+0000","last_clean":"2024-04-01T00:13:48.027883+0000","last_became_active":"2024-04-01T00:13:47.025855+0000","last_became_peered":"2024-04-01T00:13:47.025855+0000","last_unstale":"2024-04-01T00:13:48.027883+0000","last_undegraded":"2024-04-01T00:13:48.027883+0000","last_fullsized":"2024-04-01T00:13:48.027883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:56.178747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906229+0000","last_change":"2024-04-01T00:13:46.037694+0000","last_active":"2024-04-01T00:13:48.906229+0000","last_peered":"2024-04-01T00:13:48.906229+0000","last_clean":"2024-04-01T00:13:48.906229+0000","last_became_active":"2024-04-01T00:13:46.037390+0000","last_became_peered":"2024-04-01T00:13:46.037390+0000","last_unstale":"2024-04-01T00:13:48.906229+0000","last_undegraded":"2024-04-01T00:13:48.906229+0000","last_fullsized":"2024-04-01T00:13:48.906229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906550+0000","last_change":"2024-04-01T00:13:47.044544+0000","last_active":"2024-04-01T00:13:48.906550+0000","last_peered":"2024-04-01T00:13:48.906550+0000","last_clean":"2024-04-01T00:13:48.906550+0000","last_became_active":"2024-04-01T00:13:47.044266+0000","last_became_peered":"2024-04-01T00:13:47.044266+0000","last_unstale":"2024-04-01T00:13:48.906550+0000","last_undegraded":"2024-04-01T00:13:48.906550+0000","last_fullsized":"2024-04-01T00:13:48.906550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030621+0000","last_change":"2024-04-01T00:13:46.036332+0000","last_active":"2024-04-01T00:13:48.030621+0000","last_peered":"2024-04-01T00:13:48.030621+0000","last_clean":"2024-04-01T00:13:48.030621+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.030621+0000","last_undegraded":"2024-04-01T00:13:48.030621+0000","last_fullsized":"2024-04-01T00:13:48.030621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905834+0000","last_change":"2024-04-01T00:13:46.032078+0000","last_active":"2024-04-01T00:13:48.905834+0000","last_peered":"2024-04-01T00:13:48.905834+0000","last_clean":"2024-04-01T00:13:48.905834+0000","last_became_active":"2024-04-01T00:13:46.031705+0000","last_became_peered":"2024-04-01T00:13:46.031705+0000","last_unstale":"2024-04-01T00:13:48.905834+0000","last_undegraded":"2024-04-01T00:13:48.905834+0000","last_fullsized":"2024-04-01T00:13:48.905834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905625+0000","last_change":"2024-04-01T00:13:47.041444+0000","last_active":"2024-04-01T00:13:48.905625+0000","last_peered":"2024-04-01T00:13:48.905625+0000","last_clean":"2024-04-01T00:13:48.905625+0000","last_became_active":"2024-04-01T00:13:47.041163+0000","last_became_peered":"2024-04-01T00:13:47.041163+0000","last_unstale":"2024-04-01T00:13:48.905625+0000","last_undegraded":"2024-04-01T00:13:48.905625+0000","last_fullsized":"2024-04-01T00:13:48.905625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026649+0000","last_change":"2024-04-01T00:13:46.030376+0000","last_active":"2024-04-01T00:13:48.026649+0000","last_peered":"2024-04-01T00:13:48.026649+0000","last_clean":"2024-04-01T00:13:48.026649+0000","last_became_active":"2024-04-01T00:13:46.030085+0000","last_became_peered":"2024-04-01T00:13:46.030085+0000","last_unstale":"2024-04-01T00:13:48.026649+0000","last_undegraded":"2024-04-01T00:13:48.026649+0000","last_fullsized":"2024-04-01T00:13:48.026649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025414+0000","last_change":"2024-04-01T00:13:47.032819+0000","last_active":"2024-04-01T00:13:48.025414+0000","last_peered":"2024-04-01T00:13:48.025414+0000","last_clean":"2024-04-01T00:13:48.025414+0000","last_became_active":"2024-04-01T00:13:47.032501+0000","last_became_peered":"2024-04-01T00:13:47.032501+0000","last_unstale":"2024-04-01T00:13:48.025414+0000","last_undegraded":"2024-04-01T00:13:48.025414+0000","last_fullsized":"2024-04-01T00:13:48.025414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030703+0000","last_change":"2024-04-01T00:13:46.030622+0000","last_active":"2024-04-01T00:13:48.030703+0000","last_peered":"2024-04-01T00:13:48.030703+0000","last_clean":"2024-04-01T00:13:48.030703+0000","last_became_active":"2024-04-01T00:13:46.030340+0000","last_became_peered":"2024-04-01T00:13:46.030340+0000","last_unstale":"2024-04-01T00:13:48.030703+0000","last_undegraded":"2024-04-01T00:13:48.030703+0000","last_fullsized":"2024-04-01T00:13:48.030703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027187+0000","last_change":"2024-04-01T00:13:47.042195+0000","last_active":"2024-04-01T00:13:48.027187+0000","last_peered":"2024-04-01T00:13:48.027187+0000","last_clean":"2024-04-01T00:13:48.027187+0000","last_became_active":"2024-04-01T00:13:47.041925+0000","last_became_peered":"2024-04-01T00:13:47.041925+0000","last_unstale":"2024-04-01T00:13:48.027187+0000","last_undegraded":"2024-04-01T00:13:48.027187+0000","last_fullsized":"2024-04-01T00:13:48.027187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027178+0000","last_change":"2024-04-01T00:13:46.029157+0000","last_active":"2024-04-01T00:13:48.027178+0000","last_peered":"2024-04-01T00:13:48.027178+0000","last_clean":"2024-04-01T00:13:48.027178+0000","last_became_active":"2024-04-01T00:13:46.028917+0000","last_became_peered":"2024-04-01T00:13:46.028917+0000","last_unstale":"2024-04-01T00:13:48.027178+0000","last_undegraded":"2024-04-01T00:13:48.027178+0000","last_fullsized":"2024-04-01T00:13:48.027178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906652+0000","last_change":"2024-04-01T00:13:47.040888+0000","last_active":"2024-04-01T00:13:48.906652+0000","last_peered":"2024-04-01T00:13:48.906652+0000","last_clean":"2024-04-01T00:13:48.906652+0000","last_became_active":"2024-04-01T00:13:47.040621+0000","last_became_peered":"2024-04-01T00:13:47.040621+0000","last_unstale":"2024-04-01T00:13:48.906652+0000","last_undegraded":"2024-04-01T00:13:48.906652+0000","last_fullsized":"2024-04-01T00:13:48.906652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025420+0000","last_change":"2024-04-01T00:13:46.030545+0000","last_active":"2024-04-01T00:13:48.025420+0000","last_peered":"2024-04-01T00:13:48.025420+0000","last_clean":"2024-04-01T00:13:48.025420+0000","last_became_active":"2024-04-01T00:13:46.030232+0000","last_became_peered":"2024-04-01T00:13:46.030232+0000","last_unstale":"2024-04-01T00:13:48.025420+0000","last_undegraded":"2024-04-01T00:13:48.025420+0000","last_fullsized":"2024-04-01T00:13:48.025420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028260+0000","last_change":"2024-04-01T00:13:47.042059+0000","last_active":"2024-04-01T00:13:48.028260+0000","last_peered":"2024-04-01T00:13:48.028260+0000","last_clean":"2024-04-01T00:13:48.028260+0000","last_became_active":"2024-04-01T00:13:47.040047+0000","last_became_peered":"2024-04-01T00:13:47.040047+0000","last_unstale":"2024-04-01T00:13:48.028260+0000","last_undegraded":"2024-04-01T00:13:48.028260+0000","last_fullsized":"2024-04-01T00:13:48.028260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030779+0000","last_change":"2024-04-01T00:13:46.039366+0000","last_active":"2024-04-01T00:13:48.030779+0000","last_peered":"2024-04-01T00:13:48.030779+0000","last_clean":"2024-04-01T00:13:48.030779+0000","last_became_active":"2024-04-01T00:13:46.039132+0000","last_became_peered":"2024-04-01T00:13:46.039132+0000","last_unstale":"2024-04-01T00:13:48.030779+0000","last_undegraded":"2024-04-01T00:13:48.030779+0000","last_fullsized":"2024-04-01T00:13:48.030779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905125+0000","last_change":"2024-04-01T00:13:47.039118+0000","last_active":"2024-04-01T00:13:48.905125+0000","last_peered":"2024-04-01T00:13:48.905125+0000","last_clean":"2024-04-01T00:13:48.905125+0000","last_became_active":"2024-04-01T00:13:47.035193+0000","last_became_peered":"2024-04-01T00:13:47.035193+0000","last_unstale":"2024-04-01T00:13:48.905125+0000","last_undegraded":"2024-04-01T00:13:48.905125+0000","last_fullsized":"2024-04-01T00:13:48.905125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906606+0000","last_change":"2024-04-01T00:13:46.033537+0000","last_active":"2024-04-01T00:13:48.906606+0000","last_peered":"2024-04-01T00:13:48.906606+0000","last_clean":"2024-04-01T00:13:48.906606+0000","last_became_active":"2024-04-01T00:13:46.033174+0000","last_became_peered":"2024-04-01T00:13:46.033174+0000","last_unstale":"2024-04-01T00:13:48.906606+0000","last_undegraded":"2024-04-01T00:13:48.906606+0000","last_fullsized":"2024-04-01T00:13:48.906606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:15:18.515585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906163+0000","last_change":"2024-04-01T00:13:47.046668+0000","last_active":"2024-04-01T00:13:48.906163+0000","last_peered":"2024-04-01T00:13:48.906163+0000","last_clean":"2024-04-01T00:13:48.906163+0000","last_became_active":"2024-04-01T00:13:47.046387+0000","last_became_peered":"2024-04-01T00:13:47.046387+0000","last_unstale":"2024-04-01T00:13:48.906163+0000","last_undegraded":"2024-04-01T00:13:48.906163+0000","last_fullsized":"2024-04-01T00:13:48.906163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:44:37.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026803+0000","last_change":"2024-04-01T00:13:46.026873+0000","last_active":"2024-04-01T00:13:48.026803+0000","last_peered":"2024-04-01T00:13:48.026803+0000","last_clean":"2024-04-01T00:13:48.026803+0000","last_became_active":"2024-04-01T00:13:46.026564+0000","last_became_peered":"2024-04-01T00:13:46.026564+0000","last_unstale":"2024-04-01T00:13:48.026803+0000","last_undegraded":"2024-04-01T00:13:48.026803+0000","last_fullsized":"2024-04-01T00:13:48.026803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905168+0000","last_change":"2024-04-01T00:13:47.040797+0000","last_active":"2024-04-01T00:13:48.905168+0000","last_peered":"2024-04-01T00:13:48.905168+0000","last_clean":"2024-04-01T00:13:48.905168+0000","last_became_active":"2024-04-01T00:13:47.040224+0000","last_became_peered":"2024-04-01T00:13:47.040224+0000","last_unstale":"2024-04-01T00:13:48.905168+0000","last_undegraded":"2024-04-01T00:13:48.905168+0000","last_fullsized":"2024-04-01T00:13:48.905168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030476+0000","last_change":"2024-04-01T00:13:46.036254+0000","last_active":"2024-04-01T00:13:48.030476+0000","last_peered":"2024-04-01T00:13:48.030476+0000","last_clean":"2024-04-01T00:13:48.030476+0000","last_became_active":"2024-04-01T00:13:46.035969+0000","last_became_peered":"2024-04-01T00:13:46.035969+0000","last_unstale":"2024-04-01T00:13:48.030476+0000","last_undegraded":"2024-04-01T00:13:48.030476+0000","last_fullsized":"2024-04-01T00:13:48.030476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030469+0000","last_change":"2024-04-01T00:13:47.039818+0000","last_active":"2024-04-01T00:13:48.030469+0000","last_peered":"2024-04-01T00:13:48.030469+0000","last_clean":"2024-04-01T00:13:48.030469+0000","last_became_active":"2024-04-01T00:13:47.039157+0000","last_became_peered":"2024-04-01T00:13:47.039157+0000","last_unstale":"2024-04-01T00:13:48.030469+0000","last_undegraded":"2024-04-01T00:13:48.030469+0000","last_fullsized":"2024-04-01T00:13:48.030469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905588+0000","last_change":"2024-04-01T00:13:46.023660+0000","last_active":"2024-04-01T00:13:48.905588+0000","last_peered":"2024-04-01T00:13:48.905588+0000","last_clean":"2024-04-01T00:13:48.905588+0000","last_became_active":"2024-04-01T00:13:46.023368+0000","last_became_peered":"2024-04-01T00:13:46.023368+0000","last_unstale":"2024-04-01T00:13:48.905588+0000","last_undegraded":"2024-04-01T00:13:48.905588+0000","last_fullsized":"2024-04-01T00:13:48.905588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:11:34.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904995+0000","last_change":"2024-04-01T00:13:47.040468+0000","last_active":"2024-04-01T00:13:48.904995+0000","last_peered":"2024-04-01T00:13:48.904995+0000","last_clean":"2024-04-01T00:13:48.904995+0000","last_became_active":"2024-04-01T00:13:47.040198+0000","last_became_peered":"2024-04-01T00:13:47.040198+0000","last_unstale":"2024-04-01T00:13:48.904995+0000","last_undegraded":"2024-04-01T00:13:48.904995+0000","last_fullsized":"2024-04-01T00:13:48.904995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028251+0000","last_change":"2024-04-01T00:13:46.029828+0000","last_active":"2024-04-01T00:13:48.028251+0000","last_peered":"2024-04-01T00:13:48.028251+0000","last_clean":"2024-04-01T00:13:48.028251+0000","last_became_active":"2024-04-01T00:13:46.029514+0000","last_became_peered":"2024-04-01T00:13:46.029514+0000","last_unstale":"2024-04-01T00:13:48.028251+0000","last_undegraded":"2024-04-01T00:13:48.028251+0000","last_fullsized":"2024-04-01T00:13:48.028251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:16:56.432398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905327+0000","last_change":"2024-04-01T00:13:47.037521+0000","last_active":"2024-04-01T00:13:48.905327+0000","last_peered":"2024-04-01T00:13:48.905327+0000","last_clean":"2024-04-01T00:13:48.905327+0000","last_became_active":"2024-04-01T00:13:47.037208+0000","last_became_peered":"2024-04-01T00:13:47.037208+0000","last_unstale":"2024-04-01T00:13:48.905327+0000","last_undegraded":"2024-04-01T00:13:48.905327+0000","last_fullsized":"2024-04-01T00:13:48.905327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905466+0000","last_change":"2024-04-01T00:13:46.029659+0000","last_active":"2024-04-01T00:13:48.905466+0000","last_peered":"2024-04-01T00:13:48.905466+0000","last_clean":"2024-04-01T00:13:48.905466+0000","last_became_active":"2024-04-01T00:13:46.029376+0000","last_became_peered":"2024-04-01T00:13:46.029376+0000","last_unstale":"2024-04-01T00:13:48.905466+0000","last_undegraded":"2024-04-01T00:13:48.905466+0000","last_fullsized":"2024-04-01T00:13:48.905466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905503+0000","last_change":"2024-04-01T00:13:47.045515+0000","last_active":"2024-04-01T00:13:48.905503+0000","last_peered":"2024-04-01T00:13:48.905503+0000","last_clean":"2024-04-01T00:13:48.905503+0000","last_became_active":"2024-04-01T00:13:47.045138+0000","last_became_peered":"2024-04-01T00:13:47.045138+0000","last_unstale":"2024-04-01T00:13:48.905503+0000","last_undegraded":"2024-04-01T00:13:48.905503+0000","last_fullsized":"2024-04-01T00:13:48.905503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:32:58.179314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025530+0000","last_change":"2024-04-01T00:13:46.033447+0000","last_active":"2024-04-01T00:13:48.025530+0000","last_peered":"2024-04-01T00:13:48.025530+0000","last_clean":"2024-04-01T00:13:48.025530+0000","last_became_active":"2024-04-01T00:13:46.033126+0000","last_became_peered":"2024-04-01T00:13:46.033126+0000","last_unstale":"2024-04-01T00:13:48.025530+0000","last_undegraded":"2024-04-01T00:13:48.025530+0000","last_fullsized":"2024-04-01T00:13:48.025530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027072+0000","last_change":"2024-04-01T00:13:47.033643+0000","last_active":"2024-04-01T00:13:48.027072+0000","last_peered":"2024-04-01T00:13:48.027072+0000","last_clean":"2024-04-01T00:13:48.027072+0000","last_became_active":"2024-04-01T00:13:47.033350+0000","last_became_peered":"2024-04-01T00:13:47.033350+0000","last_unstale":"2024-04-01T00:13:48.027072+0000","last_undegraded":"2024-04-01T00:13:48.027072+0000","last_fullsized":"2024-04-01T00:13:48.027072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905480+0000","last_change":"2024-04-01T00:13:46.021048+0000","last_active":"2024-04-01T00:13:48.905480+0000","last_peered":"2024-04-01T00:13:48.905480+0000","last_clean":"2024-04-01T00:13:48.905480+0000","last_became_active":"2024-04-01T00:13:46.020612+0000","last_became_peered":"2024-04-01T00:13:46.020612+0000","last_unstale":"2024-04-01T00:13:48.905480+0000","last_undegraded":"2024-04-01T00:13:48.905480+0000","last_fullsized":"2024-04-01T00:13:48.905480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905112+0000","last_change":"2024-04-01T00:13:47.038059+0000","last_active":"2024-04-01T00:13:48.905112+0000","last_peered":"2024-04-01T00:13:48.905112+0000","last_clean":"2024-04-01T00:13:48.905112+0000","last_became_active":"2024-04-01T00:13:47.034213+0000","last_became_peered":"2024-04-01T00:13:47.034213+0000","last_unstale":"2024-04-01T00:13:48.905112+0000","last_undegraded":"2024-04-01T00:13:48.905112+0000","last_fullsized":"2024-04-01T00:13:48.905112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904882+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:48.904882+0000","last_peered":"2024-04-01T00:13:48.904882+0000","last_clean":"2024-04-01T00:13:48.904882+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:48.904882+0000","last_undegraded":"2024-04-01T00:13:48.904882+0000","last_fullsized":"2024-04-01T00:13:48.904882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026668+0000","last_change":"2024-04-01T00:13:46.019635+0000","last_active":"2024-04-01T00:13:48.026668+0000","last_peered":"2024-04-01T00:13:48.026668+0000","last_clean":"2024-04-01T00:13:48.026668+0000","last_became_active":"2024-04-01T00:13:46.019355+0000","last_became_peered":"2024-04-01T00:13:46.019355+0000","last_unstale":"2024-04-01T00:13:48.026668+0000","last_undegraded":"2024-04-01T00:13:48.026668+0000","last_fullsized":"2024-04-01T00:13:48.026668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906283+0000","last_change":"2024-04-01T00:13:47.034055+0000","last_active":"2024-04-01T00:13:48.906283+0000","last_peered":"2024-04-01T00:13:48.906283+0000","last_clean":"2024-04-01T00:13:48.906283+0000","last_became_active":"2024-04-01T00:13:47.033778+0000","last_became_peered":"2024-04-01T00:13:47.033778+0000","last_unstale":"2024-04-01T00:13:48.906283+0000","last_undegraded":"2024-04-01T00:13:48.906283+0000","last_fullsized":"2024-04-01T00:13:48.906283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904818+0000","last_change":"2024-04-01T00:13:46.037663+0000","last_active":"2024-04-01T00:13:48.904818+0000","last_peered":"2024-04-01T00:13:48.904818+0000","last_clean":"2024-04-01T00:13:48.904818+0000","last_became_active":"2024-04-01T00:13:46.037388+0000","last_became_peered":"2024-04-01T00:13:46.037388+0000","last_unstale":"2024-04-01T00:13:48.904818+0000","last_undegraded":"2024-04-01T00:13:48.904818+0000","last_fullsized":"2024-04-01T00:13:48.904818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906330+0000","last_change":"2024-04-01T00:13:47.038068+0000","last_active":"2024-04-01T00:13:48.906330+0000","last_peered":"2024-04-01T00:13:48.906330+0000","last_clean":"2024-04-01T00:13:48.906330+0000","last_became_active":"2024-04-01T00:13:47.037774+0000","last_became_peered":"2024-04-01T00:13:47.037774+0000","last_unstale":"2024-04-01T00:13:48.906330+0000","last_undegraded":"2024-04-01T00:13:48.906330+0000","last_fullsized":"2024-04-01T00:13:48.906330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:17:27.372297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905353+0000","last_change":"2024-04-01T00:13:46.029621+0000","last_active":"2024-04-01T00:13:48.905353+0000","last_peered":"2024-04-01T00:13:48.905353+0000","last_clean":"2024-04-01T00:13:48.905353+0000","last_became_active":"2024-04-01T00:13:46.029319+0000","last_became_peered":"2024-04-01T00:13:46.029319+0000","last_unstale":"2024-04-01T00:13:48.905353+0000","last_undegraded":"2024-04-01T00:13:48.905353+0000","last_fullsized":"2024-04-01T00:13:48.905353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025440+0000","last_change":"2024-04-01T00:13:47.030092+0000","last_active":"2024-04-01T00:13:48.025440+0000","last_peered":"2024-04-01T00:13:48.025440+0000","last_clean":"2024-04-01T00:13:48.025440+0000","last_became_active":"2024-04-01T00:13:47.029837+0000","last_became_peered":"2024-04-01T00:13:47.029837+0000","last_unstale":"2024-04-01T00:13:48.025440+0000","last_undegraded":"2024-04-01T00:13:48.025440+0000","last_fullsized":"2024-04-01T00:13:48.025440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026885+0000","last_change":"2024-04-01T00:13:46.027134+0000","last_active":"2024-04-01T00:13:48.026885+0000","last_peered":"2024-04-01T00:13:48.026885+0000","last_clean":"2024-04-01T00:13:48.026885+0000","last_became_active":"2024-04-01T00:13:46.026855+0000","last_became_peered":"2024-04-01T00:13:46.026855+0000","last_unstale":"2024-04-01T00:13:48.026885+0000","last_undegraded":"2024-04-01T00:13:48.026885+0000","last_fullsized":"2024-04-01T00:13:48.026885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028071+0000","last_change":"2024-04-01T00:13:47.041868+0000","last_active":"2024-04-01T00:13:48.028071+0000","last_peered":"2024-04-01T00:13:48.028071+0000","last_clean":"2024-04-01T00:13:48.028071+0000","last_became_active":"2024-04-01T00:13:47.040849+0000","last_became_peered":"2024-04-01T00:13:47.040849+0000","last_unstale":"2024-04-01T00:13:48.028071+0000","last_undegraded":"2024-04-01T00:13:48.028071+0000","last_fullsized":"2024-04-01T00:13:48.028071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:54.759711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025337+0000","last_change":"2024-04-01T00:13:46.026672+0000","last_active":"2024-04-01T00:13:48.025337+0000","last_peered":"2024-04-01T00:13:48.025337+0000","last_clean":"2024-04-01T00:13:48.025337+0000","last_became_active":"2024-04-01T00:13:46.026365+0000","last_became_peered":"2024-04-01T00:13:46.026365+0000","last_unstale":"2024-04-01T00:13:48.025337+0000","last_undegraded":"2024-04-01T00:13:48.025337+0000","last_fullsized":"2024-04-01T00:13:48.025337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027240+0000","last_change":"2024-04-01T00:13:47.028664+0000","last_active":"2024-04-01T00:13:48.027240+0000","last_peered":"2024-04-01T00:13:48.027240+0000","last_clean":"2024-04-01T00:13:48.027240+0000","last_became_active":"2024-04-01T00:13:47.028387+0000","last_became_peered":"2024-04-01T00:13:47.028387+0000","last_unstale":"2024-04-01T00:13:48.027240+0000","last_undegraded":"2024-04-01T00:13:48.027240+0000","last_fullsized":"2024-04-01T00:13:48.027240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905298+0000","last_change":"2024-04-01T00:13:46.037508+0000","last_active":"2024-04-01T00:13:48.905298+0000","last_peered":"2024-04-01T00:13:48.905298+0000","last_clean":"2024-04-01T00:13:48.905298+0000","last_became_active":"2024-04-01T00:13:46.037083+0000","last_became_peered":"2024-04-01T00:13:46.037083+0000","last_unstale":"2024-04-01T00:13:48.905298+0000","last_undegraded":"2024-04-01T00:13:48.905298+0000","last_fullsized":"2024-04-01T00:13:48.905298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025575+0000","last_change":"2024-04-01T00:13:47.027434+0000","last_active":"2024-04-01T00:13:48.025575+0000","last_peered":"2024-04-01T00:13:48.025575+0000","last_clean":"2024-04-01T00:13:48.025575+0000","last_became_active":"2024-04-01T00:13:47.027134+0000","last_became_peered":"2024-04-01T00:13:47.027134+0000","last_unstale":"2024-04-01T00:13:48.025575+0000","last_undegraded":"2024-04-01T00:13:48.025575+0000","last_fullsized":"2024-04-01T00:13:48.025575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030010+0000","last_change":"2024-04-01T00:13:46.029352+0000","last_active":"2024-04-01T00:13:48.030010+0000","last_peered":"2024-04-01T00:13:48.030010+0000","last_clean":"2024-04-01T00:13:48.030010+0000","last_became_active":"2024-04-01T00:13:46.029053+0000","last_became_peered":"2024-04-01T00:13:46.029053+0000","last_unstale":"2024-04-01T00:13:48.030010+0000","last_undegraded":"2024-04-01T00:13:48.030010+0000","last_fullsized":"2024-04-01T00:13:48.030010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906349+0000","last_change":"2024-04-01T00:13:47.045641+0000","last_active":"2024-04-01T00:13:48.906349+0000","last_peered":"2024-04-01T00:13:48.906349+0000","last_clean":"2024-04-01T00:13:48.906349+0000","last_became_active":"2024-04-01T00:13:47.045348+0000","last_became_peered":"2024-04-01T00:13:47.045348+0000","last_unstale":"2024-04-01T00:13:48.906349+0000","last_undegraded":"2024-04-01T00:13:48.906349+0000","last_fullsized":"2024-04-01T00:13:48.906349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030805+0000","last_change":"2024-04-01T00:13:46.037865+0000","last_active":"2024-04-01T00:13:48.030805+0000","last_peered":"2024-04-01T00:13:48.030805+0000","last_clean":"2024-04-01T00:13:48.030805+0000","last_became_active":"2024-04-01T00:13:46.037564+0000","last_became_peered":"2024-04-01T00:13:46.037564+0000","last_unstale":"2024-04-01T00:13:48.030805+0000","last_undegraded":"2024-04-01T00:13:48.030805+0000","last_fullsized":"2024-04-01T00:13:48.030805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906532+0000","last_change":"2024-04-01T00:13:47.038745+0000","last_active":"2024-04-01T00:13:48.906532+0000","last_peered":"2024-04-01T00:13:48.906532+0000","last_clean":"2024-04-01T00:13:48.906532+0000","last_became_active":"2024-04-01T00:13:47.038440+0000","last_became_peered":"2024-04-01T00:13:47.038440+0000","last_unstale":"2024-04-01T00:13:48.906532+0000","last_undegraded":"2024-04-01T00:13:48.906532+0000","last_fullsized":"2024-04-01T00:13:48.906532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904810+0000","last_change":"2024-04-01T00:13:46.037408+0000","last_active":"2024-04-01T00:13:48.904810+0000","last_peered":"2024-04-01T00:13:48.904810+0000","last_clean":"2024-04-01T00:13:48.904810+0000","last_became_active":"2024-04-01T00:13:46.037120+0000","last_became_peered":"2024-04-01T00:13:46.037120+0000","last_unstale":"2024-04-01T00:13:48.904810+0000","last_undegraded":"2024-04-01T00:13:48.904810+0000","last_fullsized":"2024-04-01T00:13:48.904810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028332+0000","last_change":"2024-04-01T00:13:47.032590+0000","last_active":"2024-04-01T00:13:48.028332+0000","last_peered":"2024-04-01T00:13:48.028332+0000","last_clean":"2024-04-01T00:13:48.028332+0000","last_became_active":"2024-04-01T00:13:47.032294+0000","last_became_peered":"2024-04-01T00:13:47.032294+0000","last_unstale":"2024-04-01T00:13:48.028332+0000","last_undegraded":"2024-04-01T00:13:48.028332+0000","last_fullsized":"2024-04-01T00:13:48.028332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025536+0000","last_change":"2024-04-01T00:13:46.036386+0000","last_active":"2024-04-01T00:13:48.025536+0000","last_peered":"2024-04-01T00:13:48.025536+0000","last_clean":"2024-04-01T00:13:48.025536+0000","last_became_active":"2024-04-01T00:13:46.036055+0000","last_became_peered":"2024-04-01T00:13:46.036055+0000","last_unstale":"2024-04-01T00:13:48.025536+0000","last_undegraded":"2024-04-01T00:13:48.025536+0000","last_fullsized":"2024-04-01T00:13:48.025536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905628+0000","last_change":"2024-04-01T00:13:47.027936+0000","last_active":"2024-04-01T00:13:48.905628+0000","last_peered":"2024-04-01T00:13:48.905628+0000","last_clean":"2024-04-01T00:13:48.905628+0000","last_became_active":"2024-04-01T00:13:47.027628+0000","last_became_peered":"2024-04-01T00:13:47.027628+0000","last_unstale":"2024-04-01T00:13:48.905628+0000","last_undegraded":"2024-04-01T00:13:48.905628+0000","last_fullsized":"2024-04-01T00:13:48.905628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025431+0000","last_change":"2024-04-01T00:13:46.028586+0000","last_active":"2024-04-01T00:13:48.025431+0000","last_peered":"2024-04-01T00:13:48.025431+0000","last_clean":"2024-04-01T00:13:48.025431+0000","last_became_active":"2024-04-01T00:13:46.028309+0000","last_became_peered":"2024-04-01T00:13:46.028309+0000","last_unstale":"2024-04-01T00:13:48.025431+0000","last_undegraded":"2024-04-01T00:13:48.025431+0000","last_fullsized":"2024-04-01T00:13:48.025431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906125+0000","last_change":"2024-04-01T00:13:47.042379+0000","last_active":"2024-04-01T00:13:48.906125+0000","last_peered":"2024-04-01T00:13:48.906125+0000","last_clean":"2024-04-01T00:13:48.906125+0000","last_became_active":"2024-04-01T00:13:47.042018+0000","last_became_peered":"2024-04-01T00:13:47.042018+0000","last_unstale":"2024-04-01T00:13:48.906125+0000","last_undegraded":"2024-04-01T00:13:48.906125+0000","last_fullsized":"2024-04-01T00:13:48.906125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905276+0000","last_change":"2024-04-01T00:13:46.037179+0000","last_active":"2024-04-01T00:13:48.905276+0000","last_peered":"2024-04-01T00:13:48.905276+0000","last_clean":"2024-04-01T00:13:48.905276+0000","last_became_active":"2024-04-01T00:13:46.036168+0000","last_became_peered":"2024-04-01T00:13:46.036168+0000","last_unstale":"2024-04-01T00:13:48.905276+0000","last_undegraded":"2024-04-01T00:13:48.905276+0000","last_fullsized":"2024-04-01T00:13:48.905276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030533+0000","last_change":"2024-04-01T00:13:47.031588+0000","last_active":"2024-04-01T00:13:48.030533+0000","last_peered":"2024-04-01T00:13:48.030533+0000","last_clean":"2024-04-01T00:13:48.030533+0000","last_became_active":"2024-04-01T00:13:47.031297+0000","last_became_peered":"2024-04-01T00:13:47.031297+0000","last_unstale":"2024-04-01T00:13:48.030533+0000","last_undegraded":"2024-04-01T00:13:48.030533+0000","last_fullsized":"2024-04-01T00:13:48.030533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027184+0000","last_change":"2024-04-01T00:13:47.036405+0000","last_active":"2024-04-01T00:13:48.027184+0000","last_peered":"2024-04-01T00:13:48.027184+0000","last_clean":"2024-04-01T00:13:48.027184+0000","last_became_active":"2024-04-01T00:13:47.036144+0000","last_became_peered":"2024-04-01T00:13:47.036144+0000","last_unstale":"2024-04-01T00:13:48.027184+0000","last_undegraded":"2024-04-01T00:13:48.027184+0000","last_fullsized":"2024-04-01T00:13:48.027184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905062+0000","last_change":"2024-04-01T00:13:46.036339+0000","last_active":"2024-04-01T00:13:48.905062+0000","last_peered":"2024-04-01T00:13:48.905062+0000","last_clean":"2024-04-01T00:13:48.905062+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.905062+0000","last_undegraded":"2024-04-01T00:13:48.905062+0000","last_fullsized":"2024-04-01T00:13:48.905062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:50.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026918+0000","last_change":"2024-04-01T00:13:47.037119+0000","last_active":"2024-04-01T00:13:48.026918+0000","last_peered":"2024-04-01T00:13:48.026918+0000","last_clean":"2024-04-01T00:13:48.026918+0000","last_became_active":"2024-04-01T00:13:47.036860+0000","last_became_peered":"2024-04-01T00:13:47.036860+0000","last_unstale":"2024-04-01T00:13:48.026918+0000","last_undegraded":"2024-04-01T00:13:48.026918+0000","last_fullsized":"2024-04-01T00:13:48.026918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905203+0000","last_change":"2024-04-01T00:13:46.036988+0000","last_active":"2024-04-01T00:13:48.905203+0000","last_peered":"2024-04-01T00:13:48.905203+0000","last_clean":"2024-04-01T00:13:48.905203+0000","last_became_active":"2024-04-01T00:13:46.036696+0000","last_became_peered":"2024-04-01T00:13:46.036696+0000","last_unstale":"2024-04-01T00:13:48.905203+0000","last_undegraded":"2024-04-01T00:13:48.905203+0000","last_fullsized":"2024-04-01T00:13:48.905203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906528+0000","last_change":"2024-04-01T00:13:47.041150+0000","last_active":"2024-04-01T00:13:48.906528+0000","last_peered":"2024-04-01T00:13:48.906528+0000","last_clean":"2024-04-01T00:13:48.906528+0000","last_became_active":"2024-04-01T00:13:47.040826+0000","last_became_peered":"2024-04-01T00:13:47.040826+0000","last_unstale":"2024-04-01T00:13:48.906528+0000","last_undegraded":"2024-04-01T00:13:48.906528+0000","last_fullsized":"2024-04-01T00:13:48.906528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906154+0000","last_change":"2024-04-01T00:13:46.032149+0000","last_active":"2024-04-01T00:13:48.906154+0000","last_peered":"2024-04-01T00:13:48.906154+0000","last_clean":"2024-04-01T00:13:48.906154+0000","last_became_active":"2024-04-01T00:13:46.031821+0000","last_became_peered":"2024-04-01T00:13:46.031821+0000","last_unstale":"2024-04-01T00:13:48.906154+0000","last_undegraded":"2024-04-01T00:13:48.906154+0000","last_fullsized":"2024-04-01T00:13:48.906154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905922+0000","last_change":"2024-04-01T00:13:47.031502+0000","last_active":"2024-04-01T00:13:48.905922+0000","last_peered":"2024-04-01T00:13:48.905922+0000","last_clean":"2024-04-01T00:13:48.905922+0000","last_became_active":"2024-04-01T00:13:47.030468+0000","last_became_peered":"2024-04-01T00:13:47.030468+0000","last_unstale":"2024-04-01T00:13:48.905922+0000","last_undegraded":"2024-04-01T00:13:48.905922+0000","last_fullsized":"2024-04-01T00:13:48.905922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906670+0000","last_change":"2024-04-01T00:13:46.038424+0000","last_active":"2024-04-01T00:13:48.906670+0000","last_peered":"2024-04-01T00:13:48.906670+0000","last_clean":"2024-04-01T00:13:48.906670+0000","last_became_active":"2024-04-01T00:13:46.038079+0000","last_became_peered":"2024-04-01T00:13:46.038079+0000","last_unstale":"2024-04-01T00:13:48.906670+0000","last_undegraded":"2024-04-01T00:13:48.906670+0000","last_fullsized":"2024-04-01T00:13:48.906670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:44:48.269461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:54.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:54.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.878+0000 7fc45546b640 1 Processor -- start 2024-04-01T00:13:54.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.878+0000 7fc45546b640 1 -- start start 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc45546b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc45015e1c0 0x7fc4500d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc45546b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4500cc340 0x7fc4500d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc45546b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4500d0060 con 0x7fc45006c390 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc45546b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fc4500d0650 con 0x7fc45015e1c0 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc45546b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc45006d100 con 0x7fc4500cc340 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc45015e1c0 0x7fc4500d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc45015e1c0 0x7fc4500d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51036/0 (socket says 172.21.15.64:51036) 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 -- 172.21.15.64:0/3193464025 learned_addr learned my addr 172.21.15.64:0/3193464025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44e7fc640 1 --2- 172.21.15.64:0/3193464025 >> v2:172.21.15.100:3300/0 conn(0x7fc4500cc340 0x7fc4500d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 -- 172.21.15.64:0/3193464025 >> v2:172.21.15.100:3300/0 conn(0x7fc4500cc340 msgr2=0x7fc4500d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 --2- 172.21.15.64:0/3193464025 >> v2:172.21.15.100:3300/0 conn(0x7fc4500cc340 0x7fc4500d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 -- 172.21.15.64:0/3193464025 >> v1:172.21.15.100:6789/0 conn(0x7fc45006c390 legacy=0x7fc45006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.879+0000 7fc44effd640 1 -- 172.21.15.64:0/3193464025 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45006d3e0 con 0x7fc45015e1c0 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.880+0000 7fc44effd640 1 --2- 172.21.15.64:0/3193464025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45015e1c0 0x7fc4500d1c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc450072e50 tx=0x7fc44802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932fea04909c5552 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.880+0000 7fc44dffb640 1 -- 172.21.15.64:0/3193464025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448035020 con 0x7fc45015e1c0 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.880+0000 7fc44dffb640 1 -- 172.21.15.64:0/3193464025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc448031870 con 0x7fc45015e1c0 2024-04-01T00:13:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.880+0000 7fc44dffb640 1 -- 172.21.15.64:0/3193464025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448031c40 con 0x7fc45015e1c0 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 -- 172.21.15.64:0/3193464025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45015e1c0 msgr2=0x7fc4500d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 --2- 172.21.15.64:0/3193464025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45015e1c0 0x7fc4500d1c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc450072e50 tx=0x7fc44802b910 comp rx=0 tx=0).stop 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 -- 172.21.15.64:0/3193464025 shutdown_connections 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 --2- 172.21.15.64:0/3193464025 >> v2:172.21.15.100:3300/0 conn(0x7fc4500cc340 0x7fc4500d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 --2- 172.21.15.64:0/3193464025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45015e1c0 0x7fc4500d1c10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 -- 172.21.15.64:0/3193464025 >> 172.21.15.64:0/3193464025 conn(0x7fc45009a980 msgr2=0x7fc45009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 -- 172.21.15.64:0/3193464025 shutdown_connections 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.881+0000 7fc45546b640 1 -- 172.21.15.64:0/3193464025 wait complete. 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.882+0000 7fc45546b640 1 Processor -- start 2024-04-01T00:13:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.882+0000 7fc45546b640 1 -- start start 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4500cc340 0x7fc45015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc45015e1c0 0x7fc4500afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4500b0b80 con 0x7fc45006c390 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4500b0a00 con 0x7fc45015e1c0 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc45546b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc4500b0d00 con 0x7fc4500cc340 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51040/0 (socket says 172.21.15.64:51040) 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 -- 172.21.15.64:0/990223567 learned_addr learned my addr 172.21.15.64:0/990223567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44f7fe640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc45015e1c0 0x7fc4500afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44e7fc640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4500cc340 0x7fc45015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 -- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4500cc340 msgr2=0x7fc45015f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4500cc340 0x7fc45015f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 -- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc45015e1c0 msgr2=0x7fc4500afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc45015e1c0 0x7fc4500afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 -- 172.21.15.64:0/990223567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc450160170 con 0x7fc45006c390 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.883+0000 7fc44effd640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc448000c00 tx=0x7fc4480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.884+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448035040 con 0x7fc45006c390 2024-04-01T00:13:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.884+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc448030d00 con 0x7fc45006c390 2024-04-01T00:13:54.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.884+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc450160450 con 0x7fc45006c390 2024-04-01T00:13:54.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.884+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480456a0 con 0x7fc45006c390 2024-04-01T00:13:54.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.884+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc450160a10 con 0x7fc45006c390 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.885+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4500681a0 con 0x7fc45006c390 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.885+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fc448038020 con 0x7fc45006c390 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.885+0000 7fc4477fe640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 0x7fc43006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.886+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc4480ac9a0 con 0x7fc45006c390 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.886+0000 7fc44e7fc640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 0x7fc43006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:54.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.886+0000 7fc44e7fc640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 0x7fc43006de50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc4501f7140 tx=0x7fc438009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:54.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:54.890+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fc44807aff0 con 0x7fc45006c390 2024-04-01T00:13:55.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.071+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc45006cb00 con 0x7fc43006b990 2024-04-01T00:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.083+0000 7fc4477fe640 1 -- 172.21.15.64:0/990223567 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7fc45006cb00 con 0x7fc43006b990 2024-04-01T00:13:55.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 msgr2=0x7fc43006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 0x7fc43006de50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc4501f7140 tx=0x7fc438009190 comp rx=0 tx=0).stop 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 msgr2=0x7fc4501f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc448000c00 tx=0x7fc4480046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 shutdown_connections 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc43006b990 0x7fc43006de50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc45015e1c0 0x7fc4500afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4500cc340 0x7fc45015f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 --2- 172.21.15.64:0/990223567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45006c390 0x7fc4501f0ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 >> 172.21.15.64:0/990223567 conn(0x7fc45009a980 msgr2=0x7fc4500b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 shutdown_connections 2024-04-01T00:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.086+0000 7fc45546b640 1 -- 172.21.15.64:0/990223567 wait complete. 2024-04-01T00:13:55.098 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-01T00:13:54.747535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001194"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905422+0000","last_change":"2024-04-01T00:13:47.045277+0000","last_active":"2024-04-01T00:13:48.905422+0000","last_peered":"2024-04-01T00:13:48.905422+0000","last_clean":"2024-04-01T00:13:48.905422+0000","last_became_active":"2024-04-01T00:13:47.044728+0000","last_became_peered":"2024-04-01T00:13:47.044728+0000","last_unstale":"2024-04-01T00:13:48.905422+0000","last_undegraded":"2024-04-01T00:13:48.905422+0000","last_fullsized":"2024-04-01T00:13:48.905422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:50:54.919900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025542+0000","last_change":"2024-04-01T00:13:47.039324+0000","last_active":"2024-04-01T00:13:48.025542+0000","last_peered":"2024-04-01T00:13:48.025542+0000","last_clean":"2024-04-01T00:13:48.025542+0000","last_became_active":"2024-04-01T00:13:47.039002+0000","last_became_peered":"2024-04-01T00:13:47.039002+0000","last_unstale":"2024-04-01T00:13:48.025542+0000","last_undegraded":"2024-04-01T00:13:48.025542+0000","last_fullsized":"2024-04-01T00:13:48.025542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906198+0000","last_change":"2024-04-01T00:13:47.039670+0000","last_active":"2024-04-01T00:13:48.906198+0000","last_peered":"2024-04-01T00:13:48.906198+0000","last_clean":"2024-04-01T00:13:48.906198+0000","last_became_active":"2024-04-01T00:13:47.037096+0000","last_became_peered":"2024-04-01T00:13:47.037096+0000","last_unstale":"2024-04-01T00:13:48.906198+0000","last_undegraded":"2024-04-01T00:13:48.906198+0000","last_fullsized":"2024-04-01T00:13:48.906198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:05:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026858+0000","last_change":"2024-04-01T00:13:47.033997+0000","last_active":"2024-04-01T00:13:48.026858+0000","last_peered":"2024-04-01T00:13:48.026858+0000","last_clean":"2024-04-01T00:13:48.026858+0000","last_became_active":"2024-04-01T00:13:47.033682+0000","last_became_peered":"2024-04-01T00:13:47.033682+0000","last_unstale":"2024-04-01T00:13:48.026858+0000","last_undegraded":"2024-04-01T00:13:48.026858+0000","last_fullsized":"2024-04-01T00:13:48.026858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905169+0000","last_change":"2024-04-01T00:13:47.040873+0000","last_active":"2024-04-01T00:13:48.905169+0000","last_peered":"2024-04-01T00:13:48.905169+0000","last_clean":"2024-04-01T00:13:48.905169+0000","last_became_active":"2024-04-01T00:13:47.039953+0000","last_became_peered":"2024-04-01T00:13:47.039953+0000","last_unstale":"2024-04-01T00:13:48.905169+0000","last_undegraded":"2024-04-01T00:13:48.905169+0000","last_fullsized":"2024-04-01T00:13:48.905169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026638+0000","last_change":"2024-04-01T00:13:47.041029+0000","last_active":"2024-04-01T00:13:48.026638+0000","last_peered":"2024-04-01T00:13:48.026638+0000","last_clean":"2024-04-01T00:13:48.026638+0000","last_became_active":"2024-04-01T00:13:47.040771+0000","last_became_peered":"2024-04-01T00:13:47.040771+0000","last_unstale":"2024-04-01T00:13:48.026638+0000","last_undegraded":"2024-04-01T00:13:48.026638+0000","last_fullsized":"2024-04-01T00:13:48.026638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904941+0000","last_change":"2024-04-01T00:13:47.042649+0000","last_active":"2024-04-01T00:13:48.904941+0000","last_peered":"2024-04-01T00:13:48.904941+0000","last_clean":"2024-04-01T00:13:48.904941+0000","last_became_active":"2024-04-01T00:13:47.042373+0000","last_became_peered":"2024-04-01T00:13:47.042373+0000","last_unstale":"2024-04-01T00:13:48.904941+0000","last_undegraded":"2024-04-01T00:13:48.904941+0000","last_fullsized":"2024-04-01T00:13:48.904941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905488+0000","last_change":"2024-04-01T00:13:47.031258+0000","last_active":"2024-04-01T00:13:48.905488+0000","last_peered":"2024-04-01T00:13:48.905488+0000","last_clean":"2024-04-01T00:13:48.905488+0000","last_became_active":"2024-04-01T00:13:47.030972+0000","last_became_peered":"2024-04-01T00:13:47.030972+0000","last_unstale":"2024-04-01T00:13:48.905488+0000","last_undegraded":"2024-04-01T00:13:48.905488+0000","last_fullsized":"2024-04-01T00:13:48.905488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905452+0000","last_change":"2024-04-01T00:13:47.045417+0000","last_active":"2024-04-01T00:13:48.905452+0000","last_peered":"2024-04-01T00:13:48.905452+0000","last_clean":"2024-04-01T00:13:48.905452+0000","last_became_active":"2024-04-01T00:13:47.044978+0000","last_became_peered":"2024-04-01T00:13:47.044978+0000","last_unstale":"2024-04-01T00:13:48.905452+0000","last_undegraded":"2024-04-01T00:13:48.905452+0000","last_fullsized":"2024-04-01T00:13:48.905452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:12:06.404460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030825+0000","last_change":"2024-04-01T00:13:47.035602+0000","last_active":"2024-04-01T00:13:48.030825+0000","last_peered":"2024-04-01T00:13:48.030825+0000","last_clean":"2024-04-01T00:13:48.030825+0000","last_became_active":"2024-04-01T00:13:47.035293+0000","last_became_peered":"2024-04-01T00:13:47.035293+0000","last_unstale":"2024-04-01T00:13:48.030825+0000","last_undegraded":"2024-04-01T00:13:48.030825+0000","last_fullsized":"2024-04-01T00:13:48.030825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030712+0000","last_change":"2024-04-01T00:13:47.037792+0000","last_active":"2024-04-01T00:13:48.030712+0000","last_peered":"2024-04-01T00:13:48.030712+0000","last_clean":"2024-04-01T00:13:48.030712+0000","last_became_active":"2024-04-01T00:13:47.037508+0000","last_became_peered":"2024-04-01T00:13:47.037508+0000","last_unstale":"2024-04-01T00:13:48.030712+0000","last_undegraded":"2024-04-01T00:13:48.030712+0000","last_fullsized":"2024-04-01T00:13:48.030712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028123+0000","last_change":"2024-04-01T00:13:47.026838+0000","last_active":"2024-04-01T00:13:48.028123+0000","last_peered":"2024-04-01T00:13:48.028123+0000","last_clean":"2024-04-01T00:13:48.028123+0000","last_became_active":"2024-04-01T00:13:47.026586+0000","last_became_peered":"2024-04-01T00:13:47.026586+0000","last_unstale":"2024-04-01T00:13:48.028123+0000","last_undegraded":"2024-04-01T00:13:48.028123+0000","last_fullsized":"2024-04-01T00:13:48.028123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:13.513798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905964+0000","last_change":"2024-04-01T00:13:47.028848+0000","last_active":"2024-04-01T00:13:48.905964+0000","last_peered":"2024-04-01T00:13:48.905964+0000","last_clean":"2024-04-01T00:13:48.905964+0000","last_became_active":"2024-04-01T00:13:47.028577+0000","last_became_peered":"2024-04-01T00:13:47.028577+0000","last_unstale":"2024-04-01T00:13:48.905964+0000","last_undegraded":"2024-04-01T00:13:48.905964+0000","last_fullsized":"2024-04-01T00:13:48.905964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906018+0000","last_change":"2024-04-01T00:13:47.039847+0000","last_active":"2024-04-01T00:13:48.906018+0000","last_peered":"2024-04-01T00:13:48.906018+0000","last_clean":"2024-04-01T00:13:48.906018+0000","last_became_active":"2024-04-01T00:13:47.039459+0000","last_became_peered":"2024-04-01T00:13:47.039459+0000","last_unstale":"2024-04-01T00:13:48.906018+0000","last_undegraded":"2024-04-01T00:13:48.906018+0000","last_fullsized":"2024-04-01T00:13:48.906018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:51.654019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025413+0000","last_change":"2024-04-01T00:13:47.034288+0000","last_active":"2024-04-01T00:13:48.025413+0000","last_peered":"2024-04-01T00:13:48.025413+0000","last_clean":"2024-04-01T00:13:48.025413+0000","last_became_active":"2024-04-01T00:13:47.034022+0000","last_became_peered":"2024-04-01T00:13:47.034022+0000","last_unstale":"2024-04-01T00:13:48.025413+0000","last_undegraded":"2024-04-01T00:13:48.025413+0000","last_fullsized":"2024-04-01T00:13:48.025413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026557+0000","last_change":"2024-04-01T00:13:47.039173+0000","last_active":"2024-04-01T00:13:48.026557+0000","last_peered":"2024-04-01T00:13:48.026557+0000","last_clean":"2024-04-01T00:13:48.026557+0000","last_became_active":"2024-04-01T00:13:47.038845+0000","last_became_peered":"2024-04-01T00:13:47.038845+0000","last_unstale":"2024-04-01T00:13:48.026557+0000","last_undegraded":"2024-04-01T00:13:48.026557+0000","last_fullsized":"2024-04-01T00:13:48.026557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027411+0000","last_change":"2024-04-01T00:13:47.043400+0000","last_active":"2024-04-01T00:13:48.027411+0000","last_peered":"2024-04-01T00:13:48.027411+0000","last_clean":"2024-04-01T00:13:48.027411+0000","last_became_active":"2024-04-01T00:13:47.043120+0000","last_became_peered":"2024-04-01T00:13:47.043120+0000","last_unstale":"2024-04-01T00:13:48.027411+0000","last_undegraded":"2024-04-01T00:13:48.027411+0000","last_fullsized":"2024-04-01T00:13:48.027411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905429+0000","last_change":"2024-04-01T00:13:47.035436+0000","last_active":"2024-04-01T00:13:48.905429+0000","last_peered":"2024-04-01T00:13:48.905429+0000","last_clean":"2024-04-01T00:13:48.905429+0000","last_became_active":"2024-04-01T00:13:47.035072+0000","last_became_peered":"2024-04-01T00:13:47.035072+0000","last_unstale":"2024-04-01T00:13:48.905429+0000","last_undegraded":"2024-04-01T00:13:48.905429+0000","last_fullsized":"2024-04-01T00:13:48.905429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905379+0000","last_change":"2024-04-01T00:13:47.040506+0000","last_active":"2024-04-01T00:13:48.905379+0000","last_peered":"2024-04-01T00:13:48.905379+0000","last_clean":"2024-04-01T00:13:48.905379+0000","last_became_active":"2024-04-01T00:13:47.040244+0000","last_became_peered":"2024-04-01T00:13:47.040244+0000","last_unstale":"2024-04-01T00:13:48.905379+0000","last_undegraded":"2024-04-01T00:13:48.905379+0000","last_fullsized":"2024-04-01T00:13:48.905379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030707+0000","last_change":"2024-04-01T00:13:47.031532+0000","last_active":"2024-04-01T00:13:48.030707+0000","last_peered":"2024-04-01T00:13:48.030707+0000","last_clean":"2024-04-01T00:13:48.030707+0000","last_became_active":"2024-04-01T00:13:47.031252+0000","last_became_peered":"2024-04-01T00:13:47.031252+0000","last_unstale":"2024-04-01T00:13:48.030707+0000","last_undegraded":"2024-04-01T00:13:48.030707+0000","last_fullsized":"2024-04-01T00:13:48.030707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906466+0000","last_change":"2024-04-01T00:13:47.047404+0000","last_active":"2024-04-01T00:13:48.906466+0000","last_peered":"2024-04-01T00:13:48.906466+0000","last_clean":"2024-04-01T00:13:48.906466+0000","last_became_active":"2024-04-01T00:13:47.047080+0000","last_became_peered":"2024-04-01T00:13:47.047080+0000","last_unstale":"2024-04-01T00:13:48.906466+0000","last_undegraded":"2024-04-01T00:13:48.906466+0000","last_fullsized":"2024-04-01T00:13:48.906466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026698+0000","last_change":"2024-04-01T00:13:47.033044+0000","last_active":"2024-04-01T00:13:48.026698+0000","last_peered":"2024-04-01T00:13:48.026698+0000","last_clean":"2024-04-01T00:13:48.026698+0000","last_became_active":"2024-04-01T00:13:47.032783+0000","last_became_peered":"2024-04-01T00:13:47.032783+0000","last_unstale":"2024-04-01T00:13:48.026698+0000","last_undegraded":"2024-04-01T00:13:48.026698+0000","last_fullsized":"2024-04-01T00:13:48.026698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905485+0000","last_change":"2024-04-01T00:13:47.035282+0000","last_active":"2024-04-01T00:13:48.905485+0000","last_peered":"2024-04-01T00:13:48.905485+0000","last_clean":"2024-04-01T00:13:48.905485+0000","last_became_active":"2024-04-01T00:13:47.034953+0000","last_became_peered":"2024-04-01T00:13:47.034953+0000","last_unstale":"2024-04-01T00:13:48.905485+0000","last_undegraded":"2024-04-01T00:13:48.905485+0000","last_fullsized":"2024-04-01T00:13:48.905485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026661+0000","last_change":"2024-04-01T00:13:47.034177+0000","last_active":"2024-04-01T00:13:48.026661+0000","last_peered":"2024-04-01T00:13:48.026661+0000","last_clean":"2024-04-01T00:13:48.026661+0000","last_became_active":"2024-04-01T00:13:47.033426+0000","last_became_peered":"2024-04-01T00:13:47.033426+0000","last_unstale":"2024-04-01T00:13:48.026661+0000","last_undegraded":"2024-04-01T00:13:48.026661+0000","last_fullsized":"2024-04-01T00:13:48.026661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906196+0000","last_change":"2024-04-01T00:13:47.034202+0000","last_active":"2024-04-01T00:13:48.906196+0000","last_peered":"2024-04-01T00:13:48.906196+0000","last_clean":"2024-04-01T00:13:48.906196+0000","last_became_active":"2024-04-01T00:13:47.033941+0000","last_became_peered":"2024-04-01T00:13:47.033941+0000","last_unstale":"2024-04-01T00:13:48.906196+0000","last_undegraded":"2024-04-01T00:13:48.906196+0000","last_fullsized":"2024-04-01T00:13:48.906196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030886+0000","last_change":"2024-04-01T00:13:47.042844+0000","last_active":"2024-04-01T00:13:48.030886+0000","last_peered":"2024-04-01T00:13:48.030886+0000","last_clean":"2024-04-01T00:13:48.030886+0000","last_became_active":"2024-04-01T00:13:47.042518+0000","last_became_peered":"2024-04-01T00:13:47.042518+0000","last_unstale":"2024-04-01T00:13:48.030886+0000","last_undegraded":"2024-04-01T00:13:48.030886+0000","last_fullsized":"2024-04-01T00:13:48.030886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025478+0000","last_change":"2024-04-01T00:13:47.032968+0000","last_active":"2024-04-01T00:13:48.025478+0000","last_peered":"2024-04-01T00:13:48.025478+0000","last_clean":"2024-04-01T00:13:48.025478+0000","last_became_active":"2024-04-01T00:13:47.032695+0000","last_became_peered":"2024-04-01T00:13:47.032695+0000","last_unstale":"2024-04-01T00:13:48.025478+0000","last_undegraded":"2024-04-01T00:13:48.025478+0000","last_fullsized":"2024-04-01T00:13:48.025478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906706+0000","last_change":"2024-04-01T00:13:47.035447+0000","last_active":"2024-04-01T00:13:48.906706+0000","last_peered":"2024-04-01T00:13:48.906706+0000","last_clean":"2024-04-01T00:13:48.906706+0000","last_became_active":"2024-04-01T00:13:47.035003+0000","last_became_peered":"2024-04-01T00:13:47.035003+0000","last_unstale":"2024-04-01T00:13:48.906706+0000","last_undegraded":"2024-04-01T00:13:48.906706+0000","last_fullsized":"2024-04-01T00:13:48.906706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906541+0000","last_change":"2024-04-01T00:13:47.039218+0000","last_active":"2024-04-01T00:13:48.906541+0000","last_peered":"2024-04-01T00:13:48.906541+0000","last_clean":"2024-04-01T00:13:48.906541+0000","last_became_active":"2024-04-01T00:13:47.037984+0000","last_became_peered":"2024-04-01T00:13:47.037984+0000","last_unstale":"2024-04-01T00:13:48.906541+0000","last_undegraded":"2024-04-01T00:13:48.906541+0000","last_fullsized":"2024-04-01T00:13:48.906541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026792+0000","last_change":"2024-04-01T00:13:47.034521+0000","last_active":"2024-04-01T00:13:48.026792+0000","last_peered":"2024-04-01T00:13:48.026792+0000","last_clean":"2024-04-01T00:13:48.026792+0000","last_became_active":"2024-04-01T00:13:47.034251+0000","last_became_peered":"2024-04-01T00:13:47.034251+0000","last_unstale":"2024-04-01T00:13:48.026792+0000","last_undegraded":"2024-04-01T00:13:48.026792+0000","last_fullsized":"2024-04-01T00:13:48.026792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905386+0000","last_change":"2024-04-01T00:13:47.035817+0000","last_active":"2024-04-01T00:13:48.905386+0000","last_peered":"2024-04-01T00:13:48.905386+0000","last_clean":"2024-04-01T00:13:48.905386+0000","last_became_active":"2024-04-01T00:13:47.034834+0000","last_became_peered":"2024-04-01T00:13:47.034834+0000","last_unstale":"2024-04-01T00:13:48.905386+0000","last_undegraded":"2024-04-01T00:13:48.905386+0000","last_fullsized":"2024-04-01T00:13:48.905386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030497+0000","last_change":"2024-04-01T00:13:47.039829+0000","last_active":"2024-04-01T00:13:48.030497+0000","last_peered":"2024-04-01T00:13:48.030497+0000","last_clean":"2024-04-01T00:13:48.030497+0000","last_became_active":"2024-04-01T00:13:47.039517+0000","last_became_peered":"2024-04-01T00:13:47.039517+0000","last_unstale":"2024-04-01T00:13:48.030497+0000","last_undegraded":"2024-04-01T00:13:48.030497+0000","last_fullsized":"2024-04-01T00:13:48.030497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027883+0000","last_change":"2024-04-01T00:13:47.026124+0000","last_active":"2024-04-01T00:13:48.027883+0000","last_peered":"2024-04-01T00:13:48.027883+0000","last_clean":"2024-04-01T00:13:48.027883+0000","last_became_active":"2024-04-01T00:13:47.025855+0000","last_became_peered":"2024-04-01T00:13:47.025855+0000","last_unstale":"2024-04-01T00:13:48.027883+0000","last_undegraded":"2024-04-01T00:13:48.027883+0000","last_fullsized":"2024-04-01T00:13:48.027883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:56.178747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906229+0000","last_change":"2024-04-01T00:13:46.037694+0000","last_active":"2024-04-01T00:13:48.906229+0000","last_peered":"2024-04-01T00:13:48.906229+0000","last_clean":"2024-04-01T00:13:48.906229+0000","last_became_active":"2024-04-01T00:13:46.037390+0000","last_became_peered":"2024-04-01T00:13:46.037390+0000","last_unstale":"2024-04-01T00:13:48.906229+0000","last_undegraded":"2024-04-01T00:13:48.906229+0000","last_fullsized":"2024-04-01T00:13:48.906229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906550+0000","last_change":"2024-04-01T00:13:47.044544+0000","last_active":"2024-04-01T00:13:48.906550+0000","last_peered":"2024-04-01T00:13:48.906550+0000","last_clean":"2024-04-01T00:13:48.906550+0000","last_became_active":"2024-04-01T00:13:47.044266+0000","last_became_peered":"2024-04-01T00:13:47.044266+0000","last_unstale":"2024-04-01T00:13:48.906550+0000","last_undegraded":"2024-04-01T00:13:48.906550+0000","last_fullsized":"2024-04-01T00:13:48.906550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030621+0000","last_change":"2024-04-01T00:13:46.036332+0000","last_active":"2024-04-01T00:13:48.030621+0000","last_peered":"2024-04-01T00:13:48.030621+0000","last_clean":"2024-04-01T00:13:48.030621+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.030621+0000","last_undegraded":"2024-04-01T00:13:48.030621+0000","last_fullsized":"2024-04-01T00:13:48.030621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905834+0000","last_change":"2024-04-01T00:13:46.032078+0000","last_active":"2024-04-01T00:13:48.905834+0000","last_peered":"2024-04-01T00:13:48.905834+0000","last_clean":"2024-04-01T00:13:48.905834+0000","last_became_active":"2024-04-01T00:13:46.031705+0000","last_became_peered":"2024-04-01T00:13:46.031705+0000","last_unstale":"2024-04-01T00:13:48.905834+0000","last_undegraded":"2024-04-01T00:13:48.905834+0000","last_fullsized":"2024-04-01T00:13:48.905834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905625+0000","last_change":"2024-04-01T00:13:47.041444+0000","last_active":"2024-04-01T00:13:48.905625+0000","last_peered":"2024-04-01T00:13:48.905625+0000","last_clean":"2024-04-01T00:13:48.905625+0000","last_became_active":"2024-04-01T00:13:47.041163+0000","last_became_peered":"2024-04-01T00:13:47.041163+0000","last_unstale":"2024-04-01T00:13:48.905625+0000","last_undegraded":"2024-04-01T00:13:48.905625+0000","last_fullsized":"2024-04-01T00:13:48.905625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026649+0000","last_change":"2024-04-01T00:13:46.030376+0000","last_active":"2024-04-01T00:13:48.026649+0000","last_peered":"2024-04-01T00:13:48.026649+0000","last_clean":"2024-04-01T00:13:48.026649+0000","last_became_active":"2024-04-01T00:13:46.030085+0000","last_became_peered":"2024-04-01T00:13:46.030085+0000","last_unstale":"2024-04-01T00:13:48.026649+0000","last_undegraded":"2024-04-01T00:13:48.026649+0000","last_fullsized":"2024-04-01T00:13:48.026649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025414+0000","last_change":"2024-04-01T00:13:47.032819+0000","last_active":"2024-04-01T00:13:48.025414+0000","last_peered":"2024-04-01T00:13:48.025414+0000","last_clean":"2024-04-01T00:13:48.025414+0000","last_became_active":"2024-04-01T00:13:47.032501+0000","last_became_peered":"2024-04-01T00:13:47.032501+0000","last_unstale":"2024-04-01T00:13:48.025414+0000","last_undegraded":"2024-04-01T00:13:48.025414+0000","last_fullsized":"2024-04-01T00:13:48.025414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030703+0000","last_change":"2024-04-01T00:13:46.030622+0000","last_active":"2024-04-01T00:13:48.030703+0000","last_peered":"2024-04-01T00:13:48.030703+0000","last_clean":"2024-04-01T00:13:48.030703+0000","last_became_active":"2024-04-01T00:13:46.030340+0000","last_became_peered":"2024-04-01T00:13:46.030340+0000","last_unstale":"2024-04-01T00:13:48.030703+0000","last_undegraded":"2024-04-01T00:13:48.030703+0000","last_fullsized":"2024-04-01T00:13:48.030703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027187+0000","last_change":"2024-04-01T00:13:47.042195+0000","last_active":"2024-04-01T00:13:48.027187+0000","last_peered":"2024-04-01T00:13:48.027187+0000","last_clean":"2024-04-01T00:13:48.027187+0000","last_became_active":"2024-04-01T00:13:47.041925+0000","last_became_peered":"2024-04-01T00:13:47.041925+0000","last_unstale":"2024-04-01T00:13:48.027187+0000","last_undegraded":"2024-04-01T00:13:48.027187+0000","last_fullsized":"2024-04-01T00:13:48.027187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027178+0000","last_change":"2024-04-01T00:13:46.029157+0000","last_active":"2024-04-01T00:13:48.027178+0000","last_peered":"2024-04-01T00:13:48.027178+0000","last_clean":"2024-04-01T00:13:48.027178+0000","last_became_active":"2024-04-01T00:13:46.028917+0000","last_became_peered":"2024-04-01T00:13:46.028917+0000","last_unstale":"2024-04-01T00:13:48.027178+0000","last_undegraded":"2024-04-01T00:13:48.027178+0000","last_fullsized":"2024-04-01T00:13:48.027178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906652+0000","last_change":"2024-04-01T00:13:47.040888+0000","last_active":"2024-04-01T00:13:48.906652+0000","last_peered":"2024-04-01T00:13:48.906652+0000","last_clean":"2024-04-01T00:13:48.906652+0000","last_became_active":"2024-04-01T00:13:47.040621+0000","last_became_peered":"2024-04-01T00:13:47.040621+0000","last_unstale":"2024-04-01T00:13:48.906652+0000","last_undegraded":"2024-04-01T00:13:48.906652+0000","last_fullsized":"2024-04-01T00:13:48.906652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025420+0000","last_change":"2024-04-01T00:13:46.030545+0000","last_active":"2024-04-01T00:13:48.025420+0000","last_peered":"2024-04-01T00:13:48.025420+0000","last_clean":"2024-04-01T00:13:48.025420+0000","last_became_active":"2024-04-01T00:13:46.030232+0000","last_became_peered":"2024-04-01T00:13:46.030232+0000","last_unstale":"2024-04-01T00:13:48.025420+0000","last_undegraded":"2024-04-01T00:13:48.025420+0000","last_fullsized":"2024-04-01T00:13:48.025420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028260+0000","last_change":"2024-04-01T00:13:47.042059+0000","last_active":"2024-04-01T00:13:48.028260+0000","last_peered":"2024-04-01T00:13:48.028260+0000","last_clean":"2024-04-01T00:13:48.028260+0000","last_became_active":"2024-04-01T00:13:47.040047+0000","last_became_peered":"2024-04-01T00:13:47.040047+0000","last_unstale":"2024-04-01T00:13:48.028260+0000","last_undegraded":"2024-04-01T00:13:48.028260+0000","last_fullsized":"2024-04-01T00:13:48.028260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030779+0000","last_change":"2024-04-01T00:13:46.039366+0000","last_active":"2024-04-01T00:13:48.030779+0000","last_peered":"2024-04-01T00:13:48.030779+0000","last_clean":"2024-04-01T00:13:48.030779+0000","last_became_active":"2024-04-01T00:13:46.039132+0000","last_became_peered":"2024-04-01T00:13:46.039132+0000","last_unstale":"2024-04-01T00:13:48.030779+0000","last_undegraded":"2024-04-01T00:13:48.030779+0000","last_fullsized":"2024-04-01T00:13:48.030779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905125+0000","last_change":"2024-04-01T00:13:47.039118+0000","last_active":"2024-04-01T00:13:48.905125+0000","last_peered":"2024-04-01T00:13:48.905125+0000","last_clean":"2024-04-01T00:13:48.905125+0000","last_became_active":"2024-04-01T00:13:47.035193+0000","last_became_peered":"2024-04-01T00:13:47.035193+0000","last_unstale":"2024-04-01T00:13:48.905125+0000","last_undegraded":"2024-04-01T00:13:48.905125+0000","last_fullsized":"2024-04-01T00:13:48.905125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906606+0000","last_change":"2024-04-01T00:13:46.033537+0000","last_active":"2024-04-01T00:13:48.906606+0000","last_peered":"2024-04-01T00:13:48.906606+0000","last_clean":"2024-04-01T00:13:48.906606+0000","last_became_active":"2024-04-01T00:13:46.033174+0000","last_became_peered":"2024-04-01T00:13:46.033174+0000","last_unstale":"2024-04-01T00:13:48.906606+0000","last_undegraded":"2024-04-01T00:13:48.906606+0000","last_fullsized":"2024-04-01T00:13:48.906606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:15:18.515585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906163+0000","last_change":"2024-04-01T00:13:47.046668+0000","last_active":"2024-04-01T00:13:48.906163+0000","last_peered":"2024-04-01T00:13:48.906163+0000","last_clean":"2024-04-01T00:13:48.906163+0000","last_became_active":"2024-04-01T00:13:47.046387+0000","last_became_peered":"2024-04-01T00:13:47.046387+0000","last_unstale":"2024-04-01T00:13:48.906163+0000","last_undegraded":"2024-04-01T00:13:48.906163+0000","last_fullsized":"2024-04-01T00:13:48.906163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:44:37.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026803+0000","last_change":"2024-04-01T00:13:46.026873+0000","last_active":"2024-04-01T00:13:48.026803+0000","last_peered":"2024-04-01T00:13:48.026803+0000","last_clean":"2024-04-01T00:13:48.026803+0000","last_became_active":"2024-04-01T00:13:46.026564+0000","last_became_peered":"2024-04-01T00:13:46.026564+0000","last_unstale":"2024-04-01T00:13:48.026803+0000","last_undegraded":"2024-04-01T00:13:48.026803+0000","last_fullsized":"2024-04-01T00:13:48.026803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905168+0000","last_change":"2024-04-01T00:13:47.040797+0000","last_active":"2024-04-01T00:13:48.905168+0000","last_peered":"2024-04-01T00:13:48.905168+0000","last_clean":"2024-04-01T00:13:48.905168+0000","last_became_active":"2024-04-01T00:13:47.040224+0000","last_became_peered":"2024-04-01T00:13:47.040224+0000","last_unstale":"2024-04-01T00:13:48.905168+0000","last_undegraded":"2024-04-01T00:13:48.905168+0000","last_fullsized":"2024-04-01T00:13:48.905168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030476+0000","last_change":"2024-04-01T00:13:46.036254+0000","last_active":"2024-04-01T00:13:48.030476+0000","last_peered":"2024-04-01T00:13:48.030476+0000","last_clean":"2024-04-01T00:13:48.030476+0000","last_became_active":"2024-04-01T00:13:46.035969+0000","last_became_peered":"2024-04-01T00:13:46.035969+0000","last_unstale":"2024-04-01T00:13:48.030476+0000","last_undegraded":"2024-04-01T00:13:48.030476+0000","last_fullsized":"2024-04-01T00:13:48.030476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030469+0000","last_change":"2024-04-01T00:13:47.039818+0000","last_active":"2024-04-01T00:13:48.030469+0000","last_peered":"2024-04-01T00:13:48.030469+0000","last_clean":"2024-04-01T00:13:48.030469+0000","last_became_active":"2024-04-01T00:13:47.039157+0000","last_became_peered":"2024-04-01T00:13:47.039157+0000","last_unstale":"2024-04-01T00:13:48.030469+0000","last_undegraded":"2024-04-01T00:13:48.030469+0000","last_fullsized":"2024-04-01T00:13:48.030469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905588+0000","last_change":"2024-04-01T00:13:46.023660+0000","last_active":"2024-04-01T00:13:48.905588+0000","last_peered":"2024-04-01T00:13:48.905588+0000","last_clean":"2024-04-01T00:13:48.905588+0000","last_became_active":"2024-04-01T00:13:46.023368+0000","last_became_peered":"2024-04-01T00:13:46.023368+0000","last_unstale":"2024-04-01T00:13:48.905588+0000","last_undegraded":"2024-04-01T00:13:48.905588+0000","last_fullsized":"2024-04-01T00:13:48.905588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:11:34.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904995+0000","last_change":"2024-04-01T00:13:47.040468+0000","last_active":"2024-04-01T00:13:48.904995+0000","last_peered":"2024-04-01T00:13:48.904995+0000","last_clean":"2024-04-01T00:13:48.904995+0000","last_became_active":"2024-04-01T00:13:47.040198+0000","last_became_peered":"2024-04-01T00:13:47.040198+0000","last_unstale":"2024-04-01T00:13:48.904995+0000","last_undegraded":"2024-04-01T00:13:48.904995+0000","last_fullsized":"2024-04-01T00:13:48.904995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028251+0000","last_change":"2024-04-01T00:13:46.029828+0000","last_active":"2024-04-01T00:13:48.028251+0000","last_peered":"2024-04-01T00:13:48.028251+0000","last_clean":"2024-04-01T00:13:48.028251+0000","last_became_active":"2024-04-01T00:13:46.029514+0000","last_became_peered":"2024-04-01T00:13:46.029514+0000","last_unstale":"2024-04-01T00:13:48.028251+0000","last_undegraded":"2024-04-01T00:13:48.028251+0000","last_fullsized":"2024-04-01T00:13:48.028251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:16:56.432398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905327+0000","last_change":"2024-04-01T00:13:47.037521+0000","last_active":"2024-04-01T00:13:48.905327+0000","last_peered":"2024-04-01T00:13:48.905327+0000","last_clean":"2024-04-01T00:13:48.905327+0000","last_became_active":"2024-04-01T00:13:47.037208+0000","last_became_peered":"2024-04-01T00:13:47.037208+0000","last_unstale":"2024-04-01T00:13:48.905327+0000","last_undegraded":"2024-04-01T00:13:48.905327+0000","last_fullsized":"2024-04-01T00:13:48.905327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905466+0000","last_change":"2024-04-01T00:13:46.029659+0000","last_active":"2024-04-01T00:13:48.905466+0000","last_peered":"2024-04-01T00:13:48.905466+0000","last_clean":"2024-04-01T00:13:48.905466+0000","last_became_active":"2024-04-01T00:13:46.029376+0000","last_became_peered":"2024-04-01T00:13:46.029376+0000","last_unstale":"2024-04-01T00:13:48.905466+0000","last_undegraded":"2024-04-01T00:13:48.905466+0000","last_fullsized":"2024-04-01T00:13:48.905466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905503+0000","last_change":"2024-04-01T00:13:47.045515+0000","last_active":"2024-04-01T00:13:48.905503+0000","last_peered":"2024-04-01T00:13:48.905503+0000","last_clean":"2024-04-01T00:13:48.905503+0000","last_became_active":"2024-04-01T00:13:47.045138+0000","last_became_peered":"2024-04-01T00:13:47.045138+0000","last_unstale":"2024-04-01T00:13:48.905503+0000","last_undegraded":"2024-04-01T00:13:48.905503+0000","last_fullsized":"2024-04-01T00:13:48.905503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:32:58.179314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025530+0000","last_change":"2024-04-01T00:13:46.033447+0000","last_active":"2024-04-01T00:13:48.025530+0000","last_peered":"2024-04-01T00:13:48.025530+0000","last_clean":"2024-04-01T00:13:48.025530+0000","last_became_active":"2024-04-01T00:13:46.033126+0000","last_became_peered":"2024-04-01T00:13:46.033126+0000","last_unstale":"2024-04-01T00:13:48.025530+0000","last_undegraded":"2024-04-01T00:13:48.025530+0000","last_fullsized":"2024-04-01T00:13:48.025530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027072+0000","last_change":"2024-04-01T00:13:47.033643+0000","last_active":"2024-04-01T00:13:48.027072+0000","last_peered":"2024-04-01T00:13:48.027072+0000","last_clean":"2024-04-01T00:13:48.027072+0000","last_became_active":"2024-04-01T00:13:47.033350+0000","last_became_peered":"2024-04-01T00:13:47.033350+0000","last_unstale":"2024-04-01T00:13:48.027072+0000","last_undegraded":"2024-04-01T00:13:48.027072+0000","last_fullsized":"2024-04-01T00:13:48.027072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905480+0000","last_change":"2024-04-01T00:13:46.021048+0000","last_active":"2024-04-01T00:13:48.905480+0000","last_peered":"2024-04-01T00:13:48.905480+0000","last_clean":"2024-04-01T00:13:48.905480+0000","last_became_active":"2024-04-01T00:13:46.020612+0000","last_became_peered":"2024-04-01T00:13:46.020612+0000","last_unstale":"2024-04-01T00:13:48.905480+0000","last_undegraded":"2024-04-01T00:13:48.905480+0000","last_fullsized":"2024-04-01T00:13:48.905480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905112+0000","last_change":"2024-04-01T00:13:47.038059+0000","last_active":"2024-04-01T00:13:48.905112+0000","last_peered":"2024-04-01T00:13:48.905112+0000","last_clean":"2024-04-01T00:13:48.905112+0000","last_became_active":"2024-04-01T00:13:47.034213+0000","last_became_peered":"2024-04-01T00:13:47.034213+0000","last_unstale":"2024-04-01T00:13:48.905112+0000","last_undegraded":"2024-04-01T00:13:48.905112+0000","last_fullsized":"2024-04-01T00:13:48.905112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904882+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:48.904882+0000","last_peered":"2024-04-01T00:13:48.904882+0000","last_clean":"2024-04-01T00:13:48.904882+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:48.904882+0000","last_undegraded":"2024-04-01T00:13:48.904882+0000","last_fullsized":"2024-04-01T00:13:48.904882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026668+0000","last_change":"2024-04-01T00:13:46.019635+0000","last_active":"2024-04-01T00:13:48.026668+0000","last_peered":"2024-04-01T00:13:48.026668+0000","last_clean":"2024-04-01T00:13:48.026668+0000","last_became_active":"2024-04-01T00:13:46.019355+0000","last_became_peered":"2024-04-01T00:13:46.019355+0000","last_unstale":"2024-04-01T00:13:48.026668+0000","last_undegraded":"2024-04-01T00:13:48.026668+0000","last_fullsized":"2024-04-01T00:13:48.026668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906283+0000","last_change":"2024-04-01T00:13:47.034055+0000","last_active":"2024-04-01T00:13:48.906283+0000","last_peered":"2024-04-01T00:13:48.906283+0000","last_clean":"2024-04-01T00:13:48.906283+0000","last_became_active":"2024-04-01T00:13:47.033778+0000","last_became_peered":"2024-04-01T00:13:47.033778+0000","last_unstale":"2024-04-01T00:13:48.906283+0000","last_undegraded":"2024-04-01T00:13:48.906283+0000","last_fullsized":"2024-04-01T00:13:48.906283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904818+0000","last_change":"2024-04-01T00:13:46.037663+0000","last_active":"2024-04-01T00:13:48.904818+0000","last_peered":"2024-04-01T00:13:48.904818+0000","last_clean":"2024-04-01T00:13:48.904818+0000","last_became_active":"2024-04-01T00:13:46.037388+0000","last_became_peered":"2024-04-01T00:13:46.037388+0000","last_unstale":"2024-04-01T00:13:48.904818+0000","last_undegraded":"2024-04-01T00:13:48.904818+0000","last_fullsized":"2024-04-01T00:13:48.904818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906330+0000","last_change":"2024-04-01T00:13:47.038068+0000","last_active":"2024-04-01T00:13:48.906330+0000","last_peered":"2024-04-01T00:13:48.906330+0000","last_clean":"2024-04-01T00:13:48.906330+0000","last_became_active":"2024-04-01T00:13:47.037774+0000","last_became_peered":"2024-04-01T00:13:47.037774+0000","last_unstale":"2024-04-01T00:13:48.906330+0000","last_undegraded":"2024-04-01T00:13:48.906330+0000","last_fullsized":"2024-04-01T00:13:48.906330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:17:27.372297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905353+0000","last_change":"2024-04-01T00:13:46.029621+0000","last_active":"2024-04-01T00:13:48.905353+0000","last_peered":"2024-04-01T00:13:48.905353+0000","last_clean":"2024-04-01T00:13:48.905353+0000","last_became_active":"2024-04-01T00:13:46.029319+0000","last_became_peered":"2024-04-01T00:13:46.029319+0000","last_unstale":"2024-04-01T00:13:48.905353+0000","last_undegraded":"2024-04-01T00:13:48.905353+0000","last_fullsized":"2024-04-01T00:13:48.905353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025440+0000","last_change":"2024-04-01T00:13:47.030092+0000","last_active":"2024-04-01T00:13:48.025440+0000","last_peered":"2024-04-01T00:13:48.025440+0000","last_clean":"2024-04-01T00:13:48.025440+0000","last_became_active":"2024-04-01T00:13:47.029837+0000","last_became_peered":"2024-04-01T00:13:47.029837+0000","last_unstale":"2024-04-01T00:13:48.025440+0000","last_undegraded":"2024-04-01T00:13:48.025440+0000","last_fullsized":"2024-04-01T00:13:48.025440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026885+0000","last_change":"2024-04-01T00:13:46.027134+0000","last_active":"2024-04-01T00:13:48.026885+0000","last_peered":"2024-04-01T00:13:48.026885+0000","last_clean":"2024-04-01T00:13:48.026885+0000","last_became_active":"2024-04-01T00:13:46.026855+0000","last_became_peered":"2024-04-01T00:13:46.026855+0000","last_unstale":"2024-04-01T00:13:48.026885+0000","last_undegraded":"2024-04-01T00:13:48.026885+0000","last_fullsized":"2024-04-01T00:13:48.026885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028071+0000","last_change":"2024-04-01T00:13:47.041868+0000","last_active":"2024-04-01T00:13:48.028071+0000","last_peered":"2024-04-01T00:13:48.028071+0000","last_clean":"2024-04-01T00:13:48.028071+0000","last_became_active":"2024-04-01T00:13:47.040849+0000","last_became_peered":"2024-04-01T00:13:47.040849+0000","last_unstale":"2024-04-01T00:13:48.028071+0000","last_undegraded":"2024-04-01T00:13:48.028071+0000","last_fullsized":"2024-04-01T00:13:48.028071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:54.759711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025337+0000","last_change":"2024-04-01T00:13:46.026672+0000","last_active":"2024-04-01T00:13:48.025337+0000","last_peered":"2024-04-01T00:13:48.025337+0000","last_clean":"2024-04-01T00:13:48.025337+0000","last_became_active":"2024-04-01T00:13:46.026365+0000","last_became_peered":"2024-04-01T00:13:46.026365+0000","last_unstale":"2024-04-01T00:13:48.025337+0000","last_undegraded":"2024-04-01T00:13:48.025337+0000","last_fullsized":"2024-04-01T00:13:48.025337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027240+0000","last_change":"2024-04-01T00:13:47.028664+0000","last_active":"2024-04-01T00:13:48.027240+0000","last_peered":"2024-04-01T00:13:48.027240+0000","last_clean":"2024-04-01T00:13:48.027240+0000","last_became_active":"2024-04-01T00:13:47.028387+0000","last_became_peered":"2024-04-01T00:13:47.028387+0000","last_unstale":"2024-04-01T00:13:48.027240+0000","last_undegraded":"2024-04-01T00:13:48.027240+0000","last_fullsized":"2024-04-01T00:13:48.027240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905298+0000","last_change":"2024-04-01T00:13:46.037508+0000","last_active":"2024-04-01T00:13:48.905298+0000","last_peered":"2024-04-01T00:13:48.905298+0000","last_clean":"2024-04-01T00:13:48.905298+0000","last_became_active":"2024-04-01T00:13:46.037083+0000","last_became_peered":"2024-04-01T00:13:46.037083+0000","last_unstale":"2024-04-01T00:13:48.905298+0000","last_undegraded":"2024-04-01T00:13:48.905298+0000","last_fullsized":"2024-04-01T00:13:48.905298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025575+0000","last_change":"2024-04-01T00:13:47.027434+0000","last_active":"2024-04-01T00:13:48.025575+0000","last_peered":"2024-04-01T00:13:48.025575+0000","last_clean":"2024-04-01T00:13:48.025575+0000","last_became_active":"2024-04-01T00:13:47.027134+0000","last_became_peered":"2024-04-01T00:13:47.027134+0000","last_unstale":"2024-04-01T00:13:48.025575+0000","last_undegraded":"2024-04-01T00:13:48.025575+0000","last_fullsized":"2024-04-01T00:13:48.025575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030010+0000","last_change":"2024-04-01T00:13:46.029352+0000","last_active":"2024-04-01T00:13:48.030010+0000","last_peered":"2024-04-01T00:13:48.030010+0000","last_clean":"2024-04-01T00:13:48.030010+0000","last_became_active":"2024-04-01T00:13:46.029053+0000","last_became_peered":"2024-04-01T00:13:46.029053+0000","last_unstale":"2024-04-01T00:13:48.030010+0000","last_undegraded":"2024-04-01T00:13:48.030010+0000","last_fullsized":"2024-04-01T00:13:48.030010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906349+0000","last_change":"2024-04-01T00:13:47.045641+0000","last_active":"2024-04-01T00:13:48.906349+0000","last_peered":"2024-04-01T00:13:48.906349+0000","last_clean":"2024-04-01T00:13:48.906349+0000","last_became_active":"2024-04-01T00:13:47.045348+0000","last_became_peered":"2024-04-01T00:13:47.045348+0000","last_unstale":"2024-04-01T00:13:48.906349+0000","last_undegraded":"2024-04-01T00:13:48.906349+0000","last_fullsized":"2024-04-01T00:13:48.906349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030805+0000","last_change":"2024-04-01T00:13:46.037865+0000","last_active":"2024-04-01T00:13:48.030805+0000","last_peered":"2024-04-01T00:13:48.030805+0000","last_clean":"2024-04-01T00:13:48.030805+0000","last_became_active":"2024-04-01T00:13:46.037564+0000","last_became_peered":"2024-04-01T00:13:46.037564+0000","last_unstale":"2024-04-01T00:13:48.030805+0000","last_undegraded":"2024-04-01T00:13:48.030805+0000","last_fullsized":"2024-04-01T00:13:48.030805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906532+0000","last_change":"2024-04-01T00:13:47.038745+0000","last_active":"2024-04-01T00:13:48.906532+0000","last_peered":"2024-04-01T00:13:48.906532+0000","last_clean":"2024-04-01T00:13:48.906532+0000","last_became_active":"2024-04-01T00:13:47.038440+0000","last_became_peered":"2024-04-01T00:13:47.038440+0000","last_unstale":"2024-04-01T00:13:48.906532+0000","last_undegraded":"2024-04-01T00:13:48.906532+0000","last_fullsized":"2024-04-01T00:13:48.906532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904810+0000","last_change":"2024-04-01T00:13:46.037408+0000","last_active":"2024-04-01T00:13:48.904810+0000","last_peered":"2024-04-01T00:13:48.904810+0000","last_clean":"2024-04-01T00:13:48.904810+0000","last_became_active":"2024-04-01T00:13:46.037120+0000","last_became_peered":"2024-04-01T00:13:46.037120+0000","last_unstale":"2024-04-01T00:13:48.904810+0000","last_undegraded":"2024-04-01T00:13:48.904810+0000","last_fullsized":"2024-04-01T00:13:48.904810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028332+0000","last_change":"2024-04-01T00:13:47.032590+0000","last_active":"2024-04-01T00:13:48.028332+0000","last_peered":"2024-04-01T00:13:48.028332+0000","last_clean":"2024-04-01T00:13:48.028332+0000","last_became_active":"2024-04-01T00:13:47.032294+0000","last_became_peered":"2024-04-01T00:13:47.032294+0000","last_unstale":"2024-04-01T00:13:48.028332+0000","last_undegraded":"2024-04-01T00:13:48.028332+0000","last_fullsized":"2024-04-01T00:13:48.028332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025536+0000","last_change":"2024-04-01T00:13:46.036386+0000","last_active":"2024-04-01T00:13:48.025536+0000","last_peered":"2024-04-01T00:13:48.025536+0000","last_clean":"2024-04-01T00:13:48.025536+0000","last_became_active":"2024-04-01T00:13:46.036055+0000","last_became_peered":"2024-04-01T00:13:46.036055+0000","last_unstale":"2024-04-01T00:13:48.025536+0000","last_undegraded":"2024-04-01T00:13:48.025536+0000","last_fullsized":"2024-04-01T00:13:48.025536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905628+0000","last_change":"2024-04-01T00:13:47.027936+0000","last_active":"2024-04-01T00:13:48.905628+0000","last_peered":"2024-04-01T00:13:48.905628+0000","last_clean":"2024-04-01T00:13:48.905628+0000","last_became_active":"2024-04-01T00:13:47.027628+0000","last_became_peered":"2024-04-01T00:13:47.027628+0000","last_unstale":"2024-04-01T00:13:48.905628+0000","last_undegraded":"2024-04-01T00:13:48.905628+0000","last_fullsized":"2024-04-01T00:13:48.905628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025431+0000","last_change":"2024-04-01T00:13:46.028586+0000","last_active":"2024-04-01T00:13:48.025431+0000","last_peered":"2024-04-01T00:13:48.025431+0000","last_clean":"2024-04-01T00:13:48.025431+0000","last_became_active":"2024-04-01T00:13:46.028309+0000","last_became_peered":"2024-04-01T00:13:46.028309+0000","last_unstale":"2024-04-01T00:13:48.025431+0000","last_undegraded":"2024-04-01T00:13:48.025431+0000","last_fullsized":"2024-04-01T00:13:48.025431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906125+0000","last_change":"2024-04-01T00:13:47.042379+0000","last_active":"2024-04-01T00:13:48.906125+0000","last_peered":"2024-04-01T00:13:48.906125+0000","last_clean":"2024-04-01T00:13:48.906125+0000","last_became_active":"2024-04-01T00:13:47.042018+0000","last_became_peered":"2024-04-01T00:13:47.042018+0000","last_unstale":"2024-04-01T00:13:48.906125+0000","last_undegraded":"2024-04-01T00:13:48.906125+0000","last_fullsized":"2024-04-01T00:13:48.906125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905276+0000","last_change":"2024-04-01T00:13:46.037179+0000","last_active":"2024-04-01T00:13:48.905276+0000","last_peered":"2024-04-01T00:13:48.905276+0000","last_clean":"2024-04-01T00:13:48.905276+0000","last_became_active":"2024-04-01T00:13:46.036168+0000","last_became_peered":"2024-04-01T00:13:46.036168+0000","last_unstale":"2024-04-01T00:13:48.905276+0000","last_undegraded":"2024-04-01T00:13:48.905276+0000","last_fullsized":"2024-04-01T00:13:48.905276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030533+0000","last_change":"2024-04-01T00:13:47.031588+0000","last_active":"2024-04-01T00:13:48.030533+0000","last_peered":"2024-04-01T00:13:48.030533+0000","last_clean":"2024-04-01T00:13:48.030533+0000","last_became_active":"2024-04-01T00:13:47.031297+0000","last_became_peered":"2024-04-01T00:13:47.031297+0000","last_unstale":"2024-04-01T00:13:48.030533+0000","last_undegraded":"2024-04-01T00:13:48.030533+0000","last_fullsized":"2024-04-01T00:13:48.030533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027184+0000","last_change":"2024-04-01T00:13:47.036405+0000","last_active":"2024-04-01T00:13:48.027184+0000","last_peered":"2024-04-01T00:13:48.027184+0000","last_clean":"2024-04-01T00:13:48.027184+0000","last_became_active":"2024-04-01T00:13:47.036144+0000","last_became_peered":"2024-04-01T00:13:47.036144+0000","last_unstale":"2024-04-01T00:13:48.027184+0000","last_undegraded":"2024-04-01T00:13:48.027184+0000","last_fullsized":"2024-04-01T00:13:48.027184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905062+0000","last_change":"2024-04-01T00:13:46.036339+0000","last_active":"2024-04-01T00:13:48.905062+0000","last_peered":"2024-04-01T00:13:48.905062+0000","last_clean":"2024-04-01T00:13:48.905062+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.905062+0000","last_undegraded":"2024-04-01T00:13:48.905062+0000","last_fullsized":"2024-04-01T00:13:48.905062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:50.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026918+0000","last_change":"2024-04-01T00:13:47.037119+0000","last_active":"2024-04-01T00:13:48.026918+0000","last_peered":"2024-04-01T00:13:48.026918+0000","last_clean":"2024-04-01T00:13:48.026918+0000","last_became_active":"2024-04-01T00:13:47.036860+0000","last_became_peered":"2024-04-01T00:13:47.036860+0000","last_unstale":"2024-04-01T00:13:48.026918+0000","last_undegraded":"2024-04-01T00:13:48.026918+0000","last_fullsized":"2024-04-01T00:13:48.026918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905203+0000","last_change":"2024-04-01T00:13:46.036988+0000","last_active":"2024-04-01T00:13:48.905203+0000","last_peered":"2024-04-01T00:13:48.905203+0000","last_clean":"2024-04-01T00:13:48.905203+0000","last_became_active":"2024-04-01T00:13:46.036696+0000","last_became_peered":"2024-04-01T00:13:46.036696+0000","last_unstale":"2024-04-01T00:13:48.905203+0000","last_undegraded":"2024-04-01T00:13:48.905203+0000","last_fullsized":"2024-04-01T00:13:48.905203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906528+0000","last_change":"2024-04-01T00:13:47.041150+0000","last_active":"2024-04-01T00:13:48.906528+0000","last_peered":"2024-04-01T00:13:48.906528+0000","last_clean":"2024-04-01T00:13:48.906528+0000","last_became_active":"2024-04-01T00:13:47.040826+0000","last_became_peered":"2024-04-01T00:13:47.040826+0000","last_unstale":"2024-04-01T00:13:48.906528+0000","last_undegraded":"2024-04-01T00:13:48.906528+0000","last_fullsized":"2024-04-01T00:13:48.906528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906154+0000","last_change":"2024-04-01T00:13:46.032149+0000","last_active":"2024-04-01T00:13:48.906154+0000","last_peered":"2024-04-01T00:13:48.906154+0000","last_clean":"2024-04-01T00:13:48.906154+0000","last_became_active":"2024-04-01T00:13:46.031821+0000","last_became_peered":"2024-04-01T00:13:46.031821+0000","last_unstale":"2024-04-01T00:13:48.906154+0000","last_undegraded":"2024-04-01T00:13:48.906154+0000","last_fullsized":"2024-04-01T00:13:48.906154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905922+0000","last_change":"2024-04-01T00:13:47.031502+0000","last_active":"2024-04-01T00:13:48.905922+0000","last_peered":"2024-04-01T00:13:48.905922+0000","last_clean":"2024-04-01T00:13:48.905922+0000","last_became_active":"2024-04-01T00:13:47.030468+0000","last_became_peered":"2024-04-01T00:13:47.030468+0000","last_unstale":"2024-04-01T00:13:48.905922+0000","last_undegraded":"2024-04-01T00:13:48.905922+0000","last_fullsized":"2024-04-01T00:13:48.905922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906670+0000","last_change":"2024-04-01T00:13:46.038424+0000","last_active":"2024-04-01T00:13:48.906670+0000","last_peered":"2024-04-01T00:13:48.906670+0000","last_clean":"2024-04-01T00:13:48.906670+0000","last_became_active":"2024-04-01T00:13:46.038079+0000","last_became_peered":"2024-04-01T00:13:46.038079+0000","last_unstale":"2024-04-01T00:13:48.906670+0000","last_undegraded":"2024-04-01T00:13:48.906670+0000","last_fullsized":"2024-04-01T00:13:48.906670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:44:48.269461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:55.107 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-01T00:13:55.107 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-01T00:13:55.108 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-01T00:13:55.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-01T00:13:55.108 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-01T00:13:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.239+0000 7f7082d56640 1 Processor -- start 2024-04-01T00:13:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.239+0000 7f7082d56640 1 -- start start 2024-04-01T00:13:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7082d56640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f707c1619e0 0x7f707c0d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7082d56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0cfae0 0x7f707c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7082d56640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707c0b3cb0 con 0x7f707c0b3820 2024-04-01T00:13:55.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7082d56640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f707c0d2ba0 con 0x7f707c1619e0 2024-04-01T00:13:55.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7082d56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f707c0d2e80 con 0x7f707c0cfae0 2024-04-01T00:13:55.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7080af2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f707c1619e0 0x7f707c0d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7080af2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f707c1619e0 0x7f707c0d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51042/0 (socket says 172.21.15.64:51042) 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f7080af2640 1 -- 172.21.15.64:0/1842718888 learned_addr learned my addr 172.21.15.64:0/1842718888 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.240+0000 7f70812f3640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0cfae0 0x7f707c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f7080af2640 1 -- 172.21.15.64:0/1842718888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0cfae0 msgr2=0x7f707c1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f7080af2640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0cfae0 0x7f707c1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f7080af2640 1 -- 172.21.15.64:0/1842718888 >> v1:172.21.15.100:6789/0 conn(0x7f707c0b3820 legacy=0x7f707c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f7080af2640 1 -- 172.21.15.64:0/1842718888 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c0ceeb0 con 0x7f707c1619e0 2024-04-01T00:13:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f70737fe640 1 -- 172.21.15.64:0/1842718888 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515334833 0 0) 0x7f707c0b3cb0 con 0x7f707c0b3820 2024-04-01T00:13:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.241+0000 7f7080af2640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0d4a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad3811dfc391d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f70737fe640 1 -- 172.21.15.64:0/1842718888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078035020 con 0x7f707c1619e0 2024-04-01T00:13:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f70737fe640 1 -- 172.21.15.64:0/1842718888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078031640 con 0x7f707c1619e0 2024-04-01T00:13:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f70737fe640 1 -- 172.21.15.64:0/1842718888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707802d040 con 0x7f707c1619e0 2024-04-01T00:13:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 -- 172.21.15.64:0/1842718888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 msgr2=0x7f707c0d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0d4a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707802b8f0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 -- 172.21.15.64:0/1842718888 shutdown_connections 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0cfae0 0x7f707c1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 --2- 172.21.15.64:0/1842718888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0d4a60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 -- 172.21.15.64:0/1842718888 >> 172.21.15.64:0/1842718888 conn(0x7f707c09d290 msgr2=0x7f707c0d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 -- 172.21.15.64:0/1842718888 shutdown_connections 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.242+0000 7f7082d56640 1 -- 172.21.15.64:0/1842718888 wait complete. 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.243+0000 7f7082d56640 1 Processor -- start 2024-04-01T00:13:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.243+0000 7f7082d56640 1 -- start start 2024-04-01T00:13:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0b3820 0x7f707c0b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f707c0cfae0 0x7f707c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f707c0b31b0 con 0x7f707c1619e0 2024-04-01T00:13:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f707c0d2890 con 0x7f707c0cfae0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7082d56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f707c0ceb10 con 0x7f707c0b3820 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f70812f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f70812f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51058/0 (socket says 172.21.15.64:51058) 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f70812f3640 1 -- 172.21.15.64:0/1129705527 learned_addr learned my addr 172.21.15.64:0/1129705527 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7080af2640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0b3820 0x7f707c0b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f7073fff640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f707c0cfae0 0x7f707c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.244+0000 7f70812f3640 1 -- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0b3820 msgr2=0x7f707c0b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70812f3640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0b3820 0x7f707c0b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70812f3640 1 -- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f707c0cfae0 msgr2=0x7f707c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70812f3640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f707c0cfae0 0x7f707c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70812f3640 1 -- 172.21.15.64:0/1129705527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c0b9ab0 con 0x7f707c1619e0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70812f3640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f707400efc0 tx=0x7f707400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074018020 con 0x7f707c1619e0 2024-04-01T00:13:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7074014480 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707c0b84d0 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074014870 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.245+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707c0b8a90 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.246+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707c161540 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.246+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7074005d10 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.247+0000 7f70717fa640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 0x7f706806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.247+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f707408b480 con 0x7f707c1619e0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.247+0000 7f7080af2640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 0x7f706806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.247+0000 7f7080af2640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 0x7f706806de50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7078036000 tx=0x7f707803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.251+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f707c161540 con 0x7f707c1619e0 2024-04-01T00:13:55.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.479+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f707c09d730 con 0x7f707c1619e0 2024-04-01T00:13:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.482+0000 7f70717fa640 1 -- 172.21.15.64:0/1129705527 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172379 (secure 0 0 0) 0x7f7074022030 con 0x7f707c1619e0 2024-04-01T00:13:55.485 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 msgr2=0x7f706806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 0x7f706806de50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7078036000 tx=0x7f707803c040 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 msgr2=0x7f707c0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f707400efc0 tx=0x7f707400c490 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 shutdown_connections 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f706806b990 0x7f706806de50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f707c1619e0 0x7f707c0b12d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f707c0cfae0 0x7f707c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 --2- 172.21.15.64:0/1129705527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f707c0b3820 0x7f707c0b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 >> 172.21.15.64:0/1129705527 conn(0x7f707c09d290 msgr2=0x7f707c0d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 shutdown_connections 2024-04-01T00:13:55.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.485+0000 7f7082d56640 1 -- 172.21.15.64:0/1129705527 wait complete. 2024-04-01T00:13:55.496 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":93206356},{"type":"v1","addr":"172.21.15.100:6833","nonce":93206356}]},"active_addr":"172.21.15.100:6833/93206356","active_change":"2024-04-01T00:13:26.736821+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4222982331}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":865606507}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2650126381}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4262622223}]}]} 2024-04-01T00:13:55.499 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-01T00:13:55.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-01T00:13:55.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:55.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.601+0000 7fa34f678640 1 Processor -- start 2024-04-01T00:13:55.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.601+0000 7fa34f678640 1 -- start start 2024-04-01T00:13:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34f678640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa34806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34f678640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa3480d2320 0x7fa34806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34f678640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3480d0060 con 0x7fa34815e1c0 2024-04-01T00:13:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34f678640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa3480d0650 con 0x7fa3480d2320 2024-04-01T00:13:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34f678640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa34806d100 con 0x7fa3480cc340 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34d414640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa34815e1c0 0x7fa3480d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47964/0 (socket says 172.21.15.64:47964) 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34d414640 1 -- 172.21.15.64:0/1824596191 learned_addr learned my addr 172.21.15.64:0/1824596191 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34dc15640 1 --2- 172.21.15.64:0/1824596191 >> v2:172.21.15.100:3300/0 conn(0x7fa3480d2320 0x7fa34806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34cc13640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa34806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa33ffff640 1 -- 172.21.15.64:0/1824596191 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709483525 0 0) 0x7fa344001110 con 0x7fa34815e1c0 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa33ffff640 1 -- 172.21.15.64:0/1824596191 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa334003520 con 0x7fa34815e1c0 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34dc15640 1 -- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 msgr2=0x7fa34806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34dc15640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa34806c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34dc15640 1 -- 172.21.15.64:0/1824596191 >> v1:172.21.15.64:6789/0 conn(0x7fa34815e1c0 legacy=0x7fa3480d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.602+0000 7fa34dc15640 1 -- 172.21.15.64:0/1824596191 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34806d3e0 con 0x7fa3480d2320 2024-04-01T00:13:55.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.603+0000 7fa34dc15640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480d2320 0x7fa34806c8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3480d07d0 tx=0x7fa34002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714353b2b14c4585 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.603+0000 7fa33ffff640 1 -- 172.21.15.64:0/1824596191 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340039070 con 0x7fa3480d2320 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.603+0000 7fa33ffff640 1 -- 172.21.15.64:0/1824596191 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34002cbb0 con 0x7fa3480d2320 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.603+0000 7fa33ffff640 1 -- 172.21.15.64:0/1824596191 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340034440 con 0x7fa3480d2320 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 -- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480d2320 msgr2=0x7fa34806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480d2320 0x7fa34806c8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3480d07d0 tx=0x7fa34002be00 comp rx=0 tx=0).stop 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 -- 172.21.15.64:0/1824596191 shutdown_connections 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa34806c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 --2- 172.21.15.64:0/1824596191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480d2320 0x7fa34806c8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 -- 172.21.15.64:0/1824596191 >> 172.21.15.64:0/1824596191 conn(0x7fa34809a980 msgr2=0x7fa34809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 -- 172.21.15.64:0/1824596191 shutdown_connections 2024-04-01T00:13:55.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.604+0000 7fa34f678640 1 -- 172.21.15.64:0/1824596191 wait complete. 2024-04-01T00:13:55.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.605+0000 7fa34f678640 1 Processor -- start 2024-04-01T00:13:55.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.605+0000 7fa34f678640 1 -- start start 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.605+0000 7fa34f678640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa348158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.605+0000 7fa34f678640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa34815e1c0 0x7fa3480c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3480b0a00 con 0x7fa3480d2320 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa34806ce10 con 0x7fa34815e1c0 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa3480b0b80 con 0x7fa3480cc340 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51072/0 (socket says 172.21.15.64:51072) 2024-04-01T00:13:55.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 -- 172.21.15.64:0/521309076 learned_addr learned my addr 172.21.15.64:0/521309076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34d414640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa348158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34dc15640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa34815e1c0 0x7fa3480c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 -- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 msgr2=0x7fa348158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa348158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 -- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa34815e1c0 msgr2=0x7fa3480c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa34815e1c0 0x7fa3480c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 -- 172.21.15.64:0/521309076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3480b5530 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34cc13640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa338009fd0 tx=0x7fa33800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338017070 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33800c940 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3480b5810 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800cd10 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.606+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3480b5dd0 con 0x7fa3480d2320 2024-04-01T00:13:55.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.607+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3480681a0 con 0x7fa3480d2320 2024-04-01T00:13:55.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.608+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fa33801a070 con 0x7fa3480d2320 2024-04-01T00:13:55.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.608+0000 7fa33dffb640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 0x7fa32c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.608+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa33808aad0 con 0x7fa3480d2320 2024-04-01T00:13:55.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.608+0000 7fa34d414640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 0x7fa32c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.609+0000 7fa34d414640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 0x7fa32c06de50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa344005880 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.612+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fa338059090 con 0x7fa3480d2320 2024-04-01T00:13:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.793+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa34815f6a0 con 0x7fa3480d2320 2024-04-01T00:13:55.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.794+0000 7fa33dffb640 1 -- 172.21.15.64:0/521309076 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fa33805d130 con 0x7fa3480d2320 2024-04-01T00:13:55.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:55.795 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 msgr2=0x7fa32c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 0x7fa32c06de50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa344005880 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 msgr2=0x7fa3480c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa338009fd0 tx=0x7fa33800ed40 comp rx=0 tx=0).stop 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 shutdown_connections 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa32c06b990 0x7fa32c06de50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa34815e1c0 0x7fa3480c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3480d2320 0x7fa3480c5820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 --2- 172.21.15.64:0/521309076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480cc340 0x7fa348158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 >> 172.21.15.64:0/521309076 conn(0x7fa34809a980 msgr2=0x7fa34806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 shutdown_connections 2024-04-01T00:13:55.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.796+0000 7fa34f678640 1 -- 172.21.15.64:0/521309076 wait complete. 2024-04-01T00:13:55.808 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-01T00:13:55.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.909+0000 7f7a250d6640 1 Processor -- start 2024-04-01T00:13:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.909+0000 7f7a250d6640 1 -- start start 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a250d6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 0x7f7a2006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a250d6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a20072b40 con 0x7f7a2006ee70 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a250d6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a200b5d20 con 0x7f7a20147420 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a250d6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7a20161e10 con 0x7f7a200b4350 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1f59e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7a2006ee70 0x7f7a20072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47966/0 (socket says 172.21.15.64:47966) 2024-04-01T00:13:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1f59e640 1 -- 172.21.15.64:0/3454690004 learned_addr learned my addr 172.21.15.64:0/3454690004 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1ed9d640 1 --2- 172.21.15.64:0/3454690004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 0x7f7a2006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1e59c640 1 -- 172.21.15.64:0/3454690004 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467294812 0 0) 0x7f7a20072b40 con 0x7f7a2006ee70 2024-04-01T00:13:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1ed9d640 1 -- 172.21.15.64:0/3454690004 >> v1:172.21.15.100:6789/0 conn(0x7f7a20147420 legacy=0x7f7a20071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1ed9d640 1 -- 172.21.15.64:0/3454690004 >> v1:172.21.15.64:6789/0 conn(0x7f7a2006ee70 legacy=0x7f7a20072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.910+0000 7f7a1ed9d640 1 -- 172.21.15.64:0/3454690004 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a201620b0 con 0x7f7a200b4350 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.911+0000 7f7a1ed9d640 1 --2- 172.21.15.64:0/3454690004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 0x7f7a2006b0c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009750 tx=0x7f7a1002b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578428d2ed19265c server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.911+0000 7f7a1e59c640 1 -- 172.21.15.64:0/3454690004 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10037070 con 0x7f7a200b4350 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.911+0000 7f7a1e59c640 1 -- 172.21.15.64:0/3454690004 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a10030710 con 0x7f7a200b4350 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.911+0000 7f7a1e59c640 1 -- 172.21.15.64:0/3454690004 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10030ae0 con 0x7f7a200b4350 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 -- 172.21.15.64:0/3454690004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 msgr2=0x7f7a2006b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 --2- 172.21.15.64:0/3454690004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 0x7f7a2006b0c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009750 tx=0x7f7a1002b730 comp rx=0 tx=0).stop 2024-04-01T00:13:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 -- 172.21.15.64:0/3454690004 shutdown_connections 2024-04-01T00:13:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 --2- 172.21.15.64:0/3454690004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a200b4350 0x7f7a2006b0c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 -- 172.21.15.64:0/3454690004 >> 172.21.15.64:0/3454690004 conn(0x7f7a200b2760 msgr2=0x7f7a200b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 -- 172.21.15.64:0/3454690004 shutdown_connections 2024-04-01T00:13:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.912+0000 7f7a250d6640 1 -- 172.21.15.64:0/3454690004 wait complete. 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.913+0000 7f7a250d6640 1 Processor -- start 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.913+0000 7f7a250d6640 1 -- start start 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a2006ee70 0x7f7a2015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a20147420 0x7f7a200d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20162950 con 0x7f7a200b4350 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20162670 con 0x7f7a20147420 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7a20147970 con 0x7f7a2006ee70 2024-04-01T00:13:55.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51080/0 (socket says 172.21.15.64:51080) 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 -- 172.21.15.64:0/1098519600 learned_addr learned my addr 172.21.15.64:0/1098519600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a1f59e640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a20147420 0x7f7a200d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a1ed9d640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a2006ee70 0x7f7a2015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 -- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a2006ee70 msgr2=0x7f7a2015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a2006ee70 0x7f7a2015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 -- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a20147420 msgr2=0x7f7a200d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a20147420 0x7f7a200d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 -- 172.21.15.64:0/1098519600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2013eab0 con 0x7f7a200b4350 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a17fff640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c7b0 tx=0x7f7a0800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08017070 con 0x7f7a200b4350 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a08005270 con 0x7f7a200b4350 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2013ed90 con 0x7f7a200b4350 2024-04-01T00:13:55.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800a640 con 0x7f7a200b4350 2024-04-01T00:13:55.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.914+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2013f350 con 0x7f7a200b4350 2024-04-01T00:13:55.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.915+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a200681a0 con 0x7f7a200b4350 2024-04-01T00:13:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.916+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7a0801a070 con 0x7f7a200b4350 2024-04-01T00:13:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.916+0000 7f7a177fe640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 0x7f7a0006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.916+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7a08089d50 con 0x7f7a200b4350 2024-04-01T00:13:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.916+0000 7f7a1ed9d640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 0x7f7a0006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.917+0000 7f7a1ed9d640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 0x7f7a0006de50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a10034000 tx=0x7f7a10008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:55.920+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f7a0808d030 con 0x7f7a200b4350 2024-04-01T00:13:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.105+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a2006f740 con 0x7f7a200b4350 2024-04-01T00:13:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.106+0000 7f7a177fe640 1 -- 172.21.15.64:0/1098519600 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f7a08058340 con 0x7f7a200b4350 2024-04-01T00:13:56.107 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:56.107 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:56.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 msgr2=0x7f7a0006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 0x7f7a0006de50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a10034000 tx=0x7f7a10008030 comp rx=0 tx=0).stop 2024-04-01T00:13:56.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 msgr2=0x7f7a200d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c7b0 tx=0x7f7a0800cc70 comp rx=0 tx=0).stop 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 shutdown_connections 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7a0006b990 0x7f7a0006de50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a20147420 0x7f7a200d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a200b4350 0x7f7a200d0d60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 --2- 172.21.15.64:0/1098519600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a2006ee70 0x7f7a2015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 >> 172.21.15.64:0/1098519600 conn(0x7f7a200b2760 msgr2=0x7f7a200b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 shutdown_connections 2024-04-01T00:13:56.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.108+0000 7f7a250d6640 1 -- 172.21.15.64:0/1098519600 wait complete. 2024-04-01T00:13:56.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-01T00:13:56.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-01T00:13:56.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-01T00:13:56.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-01T00:13:56.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-01T00:13:56.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-01T00:13:56.123 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-01T00:13:56.123 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-01T00:13:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.292+0000 7fedb9a4c640 1 Processor -- start 2024-04-01T00:13:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.292+0000 7fedb9a4c640 1 -- start start 2024-04-01T00:13:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7f416d72c640 1 Processor -- start 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7fedb9a4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 0x7fedac03b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7fedb9a4c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac009570 con 0x7fedac009190 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7fedb9a4c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac035970 con 0x7fedac048c40 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7fedb9a4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fedac035c30 con 0x7fedac049070 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb8a4a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fedac009190 0x7fedac052f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47970/0 (socket says 172.21.15.64:47970) 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb8a4a640 1 -- 172.21.15.64:0/2865084902 learned_addr learned my addr 172.21.15.64:0/2865084902 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb924b640 1 --2- 172.21.15.64:0/2865084902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 0x7fedac03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512933406 0 0) 0x7fedac035970 con 0x7fedac048c40 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda4003520 con 0x7fedac048c40 2024-04-01T00:13:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967013271 0 0) 0x7fedac009570 con 0x7fedac009190 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedac035970 con 0x7fedac009190 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb924b640 1 -- 172.21.15.64:0/2865084902 >> v1:172.21.15.100:6789/0 conn(0x7fedac048c40 legacy=0x7fedac03aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb924b640 1 -- 172.21.15.64:0/2865084902 >> v1:172.21.15.64:6789/0 conn(0x7fedac009190 legacy=0x7fedac052f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7fedb924b640 1 -- 172.21.15.64:0/2865084902 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac035f10 con 0x7fedac049070 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.293+0000 7f416d72c640 1 -- start start 2024-04-01T00:13:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7f416d72c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f416815a050 0x7f4168075170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7f416d72c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 0x7f416806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7f416d72c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416806f5a0 con 0x7f41680b6e00 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7f416d72c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f41680740d0 con 0x7f416815a050 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.294+0000 7f416d72c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f41680743b0 con 0x7f41680b5f10 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f41667fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 0x7f416806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f4167fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f41680b6e00 0x7f416806e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47972/0 (socket says 172.21.15.64:47972) 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f4167fff640 1 -- 172.21.15.64:0/3579570749 learned_addr learned my addr 172.21.15.64:0/3579570749 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f4166ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f416815a050 0x7f4168075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f11c607d640 1 Processor -- start 2024-04-01T00:13:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f11c607d640 1 -- start start 2024-04-01T00:13:56.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f1a9479a640 1 Processor -- start 2024-04-01T00:13:56.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f6529b7a640 1 Processor -- start 2024-04-01T00:13:56.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f41667fc640 1 -- 172.21.15.64:0/3579570749 >> v2:172.21.15.100:3300/0 conn(0x7f416815a050 msgr2=0x7f4168075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f41667fc640 1 --2- 172.21.15.64:0/3579570749 >> v2:172.21.15.100:3300/0 conn(0x7f416815a050 0x7f4168075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f4165ffb640 1 -- 172.21.15.64:0/3579570749 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850102492 0 0) 0x7f416806f5a0 con 0x7f41680b6e00 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f41667fc640 1 -- 172.21.15.64:0/3579570749 >> v1:172.21.15.64:6789/0 conn(0x7f41680b6e00 legacy=0x7f416806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f41667fc640 1 -- 172.21.15.64:0/3579570749 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4168074690 con 0x7f41680b5f10 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7fedb924b640 1 --2- 172.21.15.64:0/2865084902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 0x7fedac03b130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feda0009750 tx=0x7feda002b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77211033dadb0d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f6529b7a640 1 -- start start 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.295+0000 7f1a9479a640 1 -- start start 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c607d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11b8050190 0x7f11b80df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c607d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 0x7f11b8053250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c607d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b8006dc0 con 0x7f11b800aa90 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c607d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f11b8006890 con 0x7f11b8050190 2024-04-01T00:13:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c607d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f11b80070a0 con 0x7f11b80392c0 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0037070 con 0x7fedac049070 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7fedb37fe640 1 -- 172.21.15.64:0/2865084902 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda002be50 con 0x7fedac049070 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.296+0000 7f11c507b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f11b800aa90 0x7f11b804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47988/0 (socket says 172.21.15.64:47988) 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c587c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 0x7f11b8053250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c507b640 1 -- 172.21.15.64:0/3312140158 learned_addr learned my addr 172.21.15.64:0/3312140158 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c487a640 1 --2- 172.21.15.64:0/3312140158 >> v2:172.21.15.100:3300/0 conn(0x7f11b8050190 0x7f11b80df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6529b7a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65241404f0 0x7f652406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6529b7a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f65240cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6529b7a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652406e640 con 0x7f65240cdaa0 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6529b7a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f65240cdf10 con 0x7f65241404f0 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6529b7a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f65241408d0 con 0x7f65240c61c0 2024-04-01T00:13:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a9479a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a8c159fc0 0x7f1a8c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f41667fc640 1 --2- 172.21.15.64:0/3579570749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 0x7f416806b0c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41540097a0 tx=0x7f415402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2146c05ed66668 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a9479a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a8c0caf70 0x7f1a8c071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a9479a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c06f110 con 0x7f1a8c0c5880 2024-04-01T00:13:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a9479a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c071260 con 0x7f1a8c0caf70 2024-04-01T00:13:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a9479a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c0c63c0 con 0x7f1a8c159fc0 2024-04-01T00:13:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6523fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65240cdaa0 0x7f65240c4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47990/0 (socket says 172.21.15.64:47990) 2024-04-01T00:13:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f6523fff640 1 -- 172.21.15.64:0/1572684607 learned_addr learned my addr 172.21.15.64:0/1572684607 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f4165ffb640 1 -- 172.21.15.64:0/3579570749 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154031780 con 0x7f41680b5f10 2024-04-01T00:13:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f65237fe640 1 --2- 172.21.15.64:0/1572684607 >> v2:172.21.15.64:3300/0 conn(0x7f65241404f0 0x7f652406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f4165ffb640 1 -- 172.21.15.64:0/3579570749 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154037070 con 0x7f41680b5f10 2024-04-01T00:13:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a91d35640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1a8c0c5880 0x7f1a8c0ca860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:50334/0 (socket says 172.21.15.64:50334) 2024-04-01T00:13:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a91d35640 1 -- 172.21.15.64:0/1362189751 learned_addr learned my addr 172.21.15.64:0/1362189751 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f1a92536640 1 --2- 172.21.15.64:0/1362189751 >> v2:172.21.15.100:3300/0 conn(0x7f1a8c159fc0 0x7f1a8c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f6522ffd640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f65240cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c587c640 1 -- 172.21.15.64:0/3312140158 >> v2:172.21.15.100:3300/0 conn(0x7f11b8050190 msgr2=0x7f11b80df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c587c640 1 --2- 172.21.15.64:0/3312140158 >> v2:172.21.15.100:3300/0 conn(0x7f11b8050190 0x7f11b80df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f65237fe640 1 -- 172.21.15.64:0/1572684607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 msgr2=0x7f65240cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f65237fe640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f65240cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f65237fe640 1 -- 172.21.15.64:0/1572684607 >> v1:172.21.15.64:6789/0 conn(0x7f65240cdaa0 legacy=0x7f65240c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f65237fe640 1 -- 172.21.15.64:0/1572684607 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652406dda0 con 0x7f65241404f0 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2865084902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 msgr2=0x7fedac03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 --2- 172.21.15.64:0/2865084902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 0x7fedac03b130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feda0009750 tx=0x7feda002b650 comp rx=0 tx=0).stop 2024-04-01T00:13:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2865084902 shutdown_connections 2024-04-01T00:13:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 --2- 172.21.15.64:0/2865084902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac049070 0x7fedac03b130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2865084902 >> 172.21.15.64:0/2865084902 conn(0x7fedac01ef40 msgr2=0x7fedac055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2865084902 shutdown_connections 2024-04-01T00:13:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2865084902 wait complete. 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f4165ffb640 1 -- 172.21.15.64:0/3579570749 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154031780 con 0x7f41680b5f10 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 -- 172.21.15.64:0/3579570749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 msgr2=0x7f416806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 --2- 172.21.15.64:0/3579570749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 0x7f416806b0c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41540097a0 tx=0x7f415402b820 comp rx=0 tx=0).stop 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 -- 172.21.15.64:0/3579570749 shutdown_connections 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 --2- 172.21.15.64:0/3579570749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b5f10 0x7f416806b0c0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41540097a0 tx=0x7f415402b820 comp rx=0 tx=0).stop 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 --2- 172.21.15.64:0/3579570749 >> v2:172.21.15.100:3300/0 conn(0x7f416815a050 0x7f4168075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 -- 172.21.15.64:0/3579570749 >> 172.21.15.64:0/3579570749 conn(0x7f416809a980 msgr2=0x7f41680b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 -- 172.21.15.64:0/3579570749 shutdown_connections 2024-04-01T00:13:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f416d72c640 1 -- 172.21.15.64:0/3579570749 wait complete. 2024-04-01T00:13:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f1a92536640 1 -- 172.21.15.64:0/1362189751 >> v2:172.21.15.64:3300/0 conn(0x7f1a8c0caf70 msgr2=0x7f1a8c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f1a92536640 1 --2- 172.21.15.64:0/1362189751 >> v2:172.21.15.64:3300/0 conn(0x7f1a8c0caf70 0x7f1a8c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f1a92536640 1 -- 172.21.15.64:0/1362189751 >> v1:172.21.15.100:6789/0 conn(0x7f1a8c0c5880 legacy=0x7f1a8c0ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.299+0000 7f65237fe640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65241404f0 0x7f652406d4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65100097a0 tx=0x7f651002da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c466fe4e79eb0824 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.298+0000 7f1a92536640 1 -- 172.21.15.64:0/1362189751 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c15a3a0 con 0x7f1a8c159fc0 2024-04-01T00:13:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.299+0000 7f42db59e640 1 Processor -- start 2024-04-01T00:13:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.299+0000 7f1a92536640 1 --2- 172.21.15.64:0/1362189751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c5340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0cd780 tx=0x7f1a8802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921eccdb7fbdeb60 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f65227fc640 1 -- 172.21.15.64:0/1572684607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510031870 con 0x7f65241404f0 2024-04-01T00:13:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f65227fc640 1 -- 172.21.15.64:0/1572684607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6510031a10 con 0x7f65241404f0 2024-04-01T00:13:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fedb9a4c640 1 Processor -- start 2024-04-01T00:13:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c587c640 1 -- 172.21.15.64:0/3312140158 >> v1:172.21.15.64:6789/0 conn(0x7f11b800aa90 legacy=0x7f11b804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.297+0000 7f11c587c640 1 -- 172.21.15.64:0/3312140158 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b8007380 con 0x7f11b80392c0 2024-04-01T00:13:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f65227fc640 1 -- 172.21.15.64:0/1572684607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510031de0 con 0x7f65241404f0 2024-04-01T00:13:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f1a91534640 1 -- 172.21.15.64:0/1362189751 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88035020 con 0x7f1a8c159fc0 2024-04-01T00:13:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fedb9a4c640 1 -- start start 2024-04-01T00:13:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f1a91534640 1 -- 172.21.15.64:0/1362189751 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a88031640 con 0x7f1a8c159fc0 2024-04-01T00:13:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fb2a4ebe640 1 Processor -- start 2024-04-01T00:13:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f1a91534640 1 -- 172.21.15.64:0/1362189751 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8802d040 con 0x7f1a8c159fc0 2024-04-01T00:13:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fb2a4ebe640 1 -- start start 2024-04-01T00:13:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f11c587c640 1 --2- 172.21.15.64:0/3312140158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 0x7f11b8053250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11c0050720 tx=0x7f11c00a6e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1082ad2fa60419 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f416d72c640 1 Processor -- start 2024-04-01T00:13:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f11b7fff640 1 -- 172.21.15.64:0/3312140158 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c00b4070 con 0x7f11b80392c0 2024-04-01T00:13:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f11b7fff640 1 -- 172.21.15.64:0/3312140158 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c00a7aa0 con 0x7f11b80392c0 2024-04-01T00:13:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7f11b7fff640 1 -- 172.21.15.64:0/3312140158 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c00af440 con 0x7f11b80392c0 2024-04-01T00:13:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.299+0000 7f42db59e640 1 -- start start 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 -- 172.21.15.64:0/1572684607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65241404f0 msgr2=0x7f652406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65241404f0 0x7f652406d4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65100097a0 tx=0x7f651002da50 comp rx=0 tx=0).stop 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 -- 172.21.15.64:0/1572684607 shutdown_connections 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f65240cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 --2- 172.21.15.64:0/1572684607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65241404f0 0x7f652406d4a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 -- 172.21.15.64:0/1572684607 >> 172.21.15.64:0/1572684607 conn(0x7f65240cf990 msgr2=0x7f65240cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f416d72c640 1 -- start start 2024-04-01T00:13:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 -- 172.21.15.64:0/1572684607 shutdown_connections 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 -- 172.21.15.64:0/1362189751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 msgr2=0x7f1a8c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 --2- 172.21.15.64:0/1362189751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c5340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0cd780 tx=0x7f1a8802b8f0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fedb9a4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 0x7fedac0e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 -- 172.21.15.64:0/1362189751 shutdown_connections 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 --2- 172.21.15.64:0/1362189751 >> v2:172.21.15.64:3300/0 conn(0x7f1a8c0caf70 0x7f1a8c071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 --2- 172.21.15.64:0/1362189751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c5340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f1a9479a640 1 -- 172.21.15.64:0/1362189751 >> 172.21.15.64:0/1362189751 conn(0x7f1a8c0cfa90 msgr2=0x7f1a8c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fb2a4ebe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2a015e260 0x7fb2a00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fd32a981640 1 Processor -- start 2024-04-01T00:13:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.300+0000 7fd32a981640 1 -- start start 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42db59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42d40ead00 0x7f42d404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42db59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d4053720 con 0x7f42d4006ea0 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42db59e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d4053a00 con 0x7f42d404ffd0 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42db59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f42d4054330 con 0x7f42d40ead00 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fb2a4ebe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01624e0 0x7fb2a00cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42dad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f42d4006ea0 0x7f42d40382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47996/0 (socket says 172.21.15.64:47996) 2024-04-01T00:13:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f42dad9d640 1 -- 172.21.15.64:0/2783638147 learned_addr learned my addr 172.21.15.64:0/2783638147 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f11c607d640 1 -- 172.21.15.64:0/3312140158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 msgr2=0x7f11b8053250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f11c607d640 1 --2- 172.21.15.64:0/3312140158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 0x7f11b8053250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11c0050720 tx=0x7f11c00a6e40 comp rx=0 tx=0).stop 2024-04-01T00:13:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fb29f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2a015e260 0x7fb2a00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fb29f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2a015e260 0x7fb2a00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:44926/0 (socket says 172.21.15.64:44926) 2024-04-01T00:13:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7fedb8a4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 0x7fedac0e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb29f7fe640 1 -- 172.21.15.64:0/3440659789 learned_addr learned my addr 172.21.15.64:0/3440659789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/3440659789 >> v2:172.21.15.64:3300/0 conn(0x7fb2a0162940 0x7fb2a00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb8a4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 0x7fedac0e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51106/0 (socket says 172.21.15.64:51106) 2024-04-01T00:13:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb8a4a640 1 -- 172.21.15.64:0/2582559424 learned_addr learned my addr 172.21.15.64:0/2582559424 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd32a981640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd31c04fc40 0x7fd31c0517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd32a981640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd31c0081a0 0x7fd31c050020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb9a4c640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac048c40 0x7fedac0dc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f1a9479a640 1 -- 172.21.15.64:0/1362189751 shutdown_connections 2024-04-01T00:13:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb29ffff640 1 --2- 172.21.15.64:0/3440659789 >> v2:172.21.15.64:3300/0 conn(0x7fb2a0162940 0x7fb2a00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a006efb0 con 0x7fb2a0162940 2024-04-01T00:13:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a00ce1d0 con 0x7fb2a015e260 2024-04-01T00:13:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.301+0000 7f6529b7a640 1 -- 172.21.15.64:0/1572684607 wait complete. 2024-04-01T00:13:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a00ce4b0 con 0x7fb2a01624e0 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fb29effd640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01624e0 0x7fb2a00cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336619069 0 0) 0x7f42d4053720 con 0x7f42d4006ea0 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f42da59c640 1 --2- 172.21.15.64:0/2783638147 >> v2:172.21.15.64:3300/0 conn(0x7f42d40ead00 0x7f42d404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c4003520 con 0x7f42d4006ea0 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f416d72c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b6e00 0x7f41680cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852088437 0 0) 0x7f42d4053a00 con 0x7f42d404ffd0 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d4053720 con 0x7f42d404ffd0 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f11c607d640 1 -- 172.21.15.64:0/3312140158 shutdown_connections 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f11c607d640 1 --2- 172.21.15.64:0/3312140158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b80392c0 0x7f11b8053250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f11c607d640 1 --2- 172.21.15.64:0/3312140158 >> v2:172.21.15.100:3300/0 conn(0x7f11b8050190 0x7f11b80df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f11c607d640 1 -- 172.21.15.64:0/3312140158 >> 172.21.15.64:0/3312140158 conn(0x7f11b8009fa0 msgr2=0x7f11b800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb9a4c640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedac049070 0x7fedac033880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0367b0 con 0x7fedac009190 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0364d0 con 0x7fedac049070 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0540a0 con 0x7fedac048c40 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd32a981640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c048550 con 0x7fd31c050560 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd32a981640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c0487f0 con 0x7fd31c0081a0 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd32a981640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c048ab0 con 0x7fd31c04fc40 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd3237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd31c0081a0 0x7fd31c050020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd323fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd31c04fc40 0x7fd31c0517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd3237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd31c0081a0 0x7fd31c050020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51134/0 (socket says 172.21.15.64:51134) 2024-04-01T00:13:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd3237fe640 1 -- 172.21.15.64:0/2127013792 learned_addr learned my addr 172.21.15.64:0/2127013792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f416d72c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416815a050 0x7f41681f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f416d72c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681e3610 0x7f416815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f416d72c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f416815a430 con 0x7f416815a050 2024-04-01T00:13:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f4166ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b6e00 0x7f41680cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f6529b7a640 1 Processor -- start 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f6529b7a640 1 -- start start 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f416d72c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4168074c50 con 0x7f41681e3610 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f416d72c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f416815a5b0 con 0x7f41680b6e00 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f4167fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681e3610 0x7f416815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f4167fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681e3610 0x7f416815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:44948/0 (socket says 172.21.15.64:44948) 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f4167fff640 1 -- 172.21.15.64:0/1779886327 learned_addr learned my addr 172.21.15.64:0/1779886327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f41667fc640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416815a050 0x7f41681f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131395388 0 0) 0x7fd31c048550 con 0x7fd31c050560 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd314003520 con 0x7fd31c050560 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664191806 0 0) 0x7fd314003520 con 0x7fd31c050560 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 >> v2:172.21.15.100:3300/0 conn(0x7fd31c04fc40 msgr2=0x7fd31c0517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 --2- 172.21.15.64:0/2127013792 >> v2:172.21.15.100:3300/0 conn(0x7fd31c04fc40 0x7fd31c0517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 >> v2:172.21.15.64:3300/0 conn(0x7fd31c0081a0 msgr2=0x7fd31c050020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 --2- 172.21.15.64:0/2127013792 >> v2:172.21.15.64:3300/0 conn(0x7fd31c0081a0 0x7fd31c050020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0530d0 con 0x7fd31c050560 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fd32404fd00 con 0x7fd31c050560 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 >> v1:172.21.15.100:6789/0 conn(0x7fd31c050560 legacy=0x7fd31c0538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314004350 0x7fd314004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd314004e40 0x7fd31400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0487f0 con 0x7fd314004350 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c048ab0 con 0x7fd3140039b0 2024-04-01T00:13:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd31400bf00 con 0x7fd314004e40 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd323fff640 1 --2- 172.21.15.64:0/2127013792 >> v2:172.21.15.100:3300/0 conn(0x7fd31c04fc40 0x7fd31c0517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3048128275 0 0) 0x7f42d4053720 con 0x7f42d404ffd0 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 >> v2:172.21.15.64:3300/0 conn(0x7f42d40ead00 msgr2=0x7f42d404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 --2- 172.21.15.64:0/2783638147 >> v2:172.21.15.64:3300/0 conn(0x7f42d40ead00 0x7f42d404fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 >> v1:172.21.15.64:6789/0 conn(0x7f42d4006ea0 legacy=0x7f42d40382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d4053ce0 con 0x7f42d404ffd0 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f42cc001670 con 0x7f42d404ffd0 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 >> v1:172.21.15.100:6789/0 conn(0x7f42d404ffd0 legacy=0x7f42d4037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42c4003b40 0x7f42c4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c4004800 0x7f42c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 0x7f42c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4054330 con 0x7f42c4003b40 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42c400bf60 con 0x7f42c4004800 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f42c400c160 con 0x7f42c40051c0 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fb29f7fe640 1 -- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01624e0 msgr2=0x7fb2a00cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01624e0 0x7fb2a00cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fb29f7fe640 1 -- 172.21.15.64:0/3440659789 >> v2:172.21.15.64:3300/0 conn(0x7fb2a0162940 msgr2=0x7fb2a00ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/3440659789 >> v2:172.21.15.64:3300/0 conn(0x7fb2a0162940 0x7fb2a00ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fb29f7fe640 1 -- 172.21.15.64:0/3440659789 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a00ce790 con 0x7fb2a015e260 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb3fff640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac048c40 0x7fedac0dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb924b640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedac049070 0x7fedac033880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb8a4a640 1 -- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac048c40 msgr2=0x7fedac0dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb8a4a640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedac048c40 0x7fedac0dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb8a4a640 1 -- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedac049070 msgr2=0x7fedac033880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb8a4a640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedac049070 0x7fedac033880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fedb8a4a640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac0ebe50 con 0x7fedac009190 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7fd328f1e640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd314004e40 0x7fd31400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42dad9d640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 0x7f42c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.303+0000 7f42d9d9b640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c4004800 0x7f42c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.302+0000 7f1a9479a640 1 -- 172.21.15.64:0/1362189751 wait complete. 2024-04-01T00:13:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42da59c640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42c4003b40 0x7f42c4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd3237fe640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314004350 0x7fd314004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f1a9479a640 1 Processor -- start 2024-04-01T00:13:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a00cecb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006bcf0 tx=0x7fb29002b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167cf482f13e0f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f11c607d640 1 -- 172.21.15.64:0/3312140158 shutdown_connections 2024-04-01T00:13:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 -- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd314004e40 msgr2=0x7fd31400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f652415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65240cdaa0 0x7f65240b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd314004e40 0x7fd31400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f41667fc640 1 -- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b6e00 msgr2=0x7f41680cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f41667fc640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b6e00 0x7f41680cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f41667fc640 1 -- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681e3610 msgr2=0x7f416815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f41667fc640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681e3610 0x7f416815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f41667fc640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681e5070 con 0x7f416815a050 2024-04-01T00:13:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f1a9479a640 1 -- start start 2024-04-01T00:13:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f4166ffd640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41680b6e00 0x7f41680cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 -- 172.21.15.64:0/2127013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314004350 msgr2=0x7fd314004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65241404f0 0x7f65240b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f652406d9e0 con 0x7f65240cdaa0 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f652406e360 con 0x7f65241404f0 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6529b7a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f652406db60 con 0x7f65240c61c0 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65240cdaa0 0x7f65240b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65240cdaa0 0x7f65240b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51164/0 (socket says 172.21.15.64:51164) 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f6522ffd640 1 -- 172.21.15.64:0/3446560292 learned_addr learned my addr 172.21.15.64:0/3446560292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f65237fe640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f652415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314004350 0x7fd314004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd323fff640 1 -- 172.21.15.64:0/2127013792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0530d0 con 0x7fd3140039b0 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42dad9d640 1 -- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c4004800 msgr2=0x7f42c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42dad9d640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c4004800 0x7f42c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42dad9d640 1 -- 172.21.15.64:0/2783638147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42c4003b40 msgr2=0x7f42c4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42dad9d640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42c4003b40 0x7f42c4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42dad9d640 1 -- 172.21.15.64:0/2783638147 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d4053ce0 con 0x7f42c40051c0 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42da59c640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42c4003b40 0x7f42c4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7fd328f1e640 1 -- 172.21.15.64:0/2127013792 reap_dead start 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f42d9d9b640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c4004800 0x7f42c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f6523fff640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65241404f0 0x7f65240b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 -- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65241404f0 msgr2=0x7f65240b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65241404f0 0x7f65240b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 -- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65240cdaa0 msgr2=0x7f65240b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65240cdaa0 0x7f65240b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f42dad9d640 1 -- 172.21.15.64:0/2783638147 reap_dead start 2024-04-01T00:13:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fedb8a4a640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 0x7fedac0e5450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fedb4050040 tx=0x7fedb402d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652415a6a0 con 0x7f65240c61c0 2024-04-01T00:13:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f41667fc640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416815a050 0x7f41681f85b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4168074530 tx=0x7f41540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f42dad9d640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 0x7f42c400b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f42dc050720 tx=0x7f42c4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0dd6bd50e6cf8c server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154006e10 con 0x7f416815a050 2024-04-01T00:13:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154031580 con 0x7f416815a050 2024-04-01T00:13:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681e3fd0 con 0x7f416815a050 2024-04-01T00:13:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.304+0000 7f11c607d640 1 -- 172.21.15.64:0/3312140158 wait complete. 2024-04-01T00:13:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154031950 con 0x7f416815a050 2024-04-01T00:13:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0b4070 con 0x7f42c40051c0 2024-04-01T00:13:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc0a7b30 con 0x7f42c40051c0 2024-04-01T00:13:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f42d959a640 1 -- 172.21.15.64:0/2783638147 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0503a0 con 0x7f42c40051c0 2024-04-01T00:13:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f65237fe640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f652415c240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f65100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c0c5880 0x7f1a8c146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a8c0caf70 0x7f1a8c0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681e4870 con 0x7f416815a050 2024-04-01T00:13:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fb29e7fc640 1 -- 172.21.15.64:0/3440659789 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290034070 con 0x7fb2a015e260 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fb29e7fc640 1 -- 172.21.15.64:0/3440659789 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290030870 con 0x7fb2a015e260 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fb29e7fc640 1 -- 172.21.15.64:0/3440659789 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290030c80 con 0x7fb2a015e260 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c071490 con 0x7f1a8c0c5880 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c0c60e0 con 0x7f1a8c159fc0 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7f1a9479a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c071770 con 0x7f1a8c0caf70 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb40734d0 con 0x7fedac009190 2024-04-01T00:13:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb4073690 con 0x7fedac009190 2024-04-01T00:13:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4071970 con 0x7fedac009190 2024-04-01T00:13:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.305+0000 7fd323fff640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd318009c90 tx=0x7fd314005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7accbe88467178 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac0e4260 con 0x7fedac009190 2024-04-01T00:13:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510047020 con 0x7f65240c61c0 2024-04-01T00:13:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f11c607d640 1 Processor -- start 2024-04-01T00:13:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652415a960 con 0x7f65240c61c0 2024-04-01T00:13:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f652415af20 con 0x7f65240c61c0 2024-04-01T00:13:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f42db59e640 1 -- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 msgr2=0x7f42c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f42db59e640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 0x7f42c400b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f42dc050720 tx=0x7f42c4005830 comp rx=0 tx=0).stop 2024-04-01T00:13:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 msgr2=0x7fb2a00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a00cecb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006bcf0 tx=0x7fb29002b9b0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f1a92536640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c0c5880 0x7f1a8c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f1a92536640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c0c5880 0x7f1a8c146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51174/0 (socket says 172.21.15.64:51174) 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f1a92536640 1 -- 172.21.15.64:0/1593583224 learned_addr learned my addr 172.21.15.64:0/1593583224 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f1a91d35640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a8c0caf70 0x7f1a8c0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a92d37640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f42db59e640 1 -- 172.21.15.64:0/2783638147 shutdown_connections 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f42db59e640 1 --2- 172.21.15.64:0/2783638147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42c40051c0 0x7f42c400b9c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f42db59e640 1 -- 172.21.15.64:0/2783638147 >> 172.21.15.64:0/2783638147 conn(0x7f42d4009fb0 msgr2=0x7f42d400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6510039800 con 0x7f65240c61c0 2024-04-01T00:13:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7f11c607d640 1 -- start start 2024-04-01T00:13:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510047020 con 0x7f65240c61c0 2024-04-01T00:13:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b800aa90 0x7f11b804e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fedb9a4c640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac0ebb20 con 0x7fedac009190 2024-04-01T00:13:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 0x7f11b81897c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f42db59e640 1 -- 172.21.15.64:0/2783638147 shutdown_connections 2024-04-01T00:13:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41680681a0 con 0x7f416815a050 2024-04-01T00:13:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8050190 0x7f11b8189d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 -- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 msgr2=0x7f1a8c0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8c159fc0 0x7f1a8c0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 -- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c0c5880 msgr2=0x7f1a8c146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c0c5880 0x7f1a8c146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c146da0 con 0x7f1a8c0caf70 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318039070 con 0x7fd3140039b0 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31802c910 con 0x7fd3140039b0 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.306+0000 7fd322ffd640 1 -- 172.21.15.64:0/2127013792 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318009860 con 0x7fd3140039b0 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 -- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 msgr2=0x7fd314003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd318009c90 tx=0x7fd314005830 comp rx=0 tx=0).stop 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 -- 172.21.15.64:0/2127013792 shutdown_connections 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 --2- 172.21.15.64:0/2127013792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 -- 172.21.15.64:0/2127013792 >> 172.21.15.64:0/2127013792 conn(0x7fd31c055380 msgr2=0x7fd31c055790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c487a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 0x7f11b81897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c487a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 0x7f11b81897c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45004/0 (socket says 172.21.15.64:45004) 2024-04-01T00:13:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f652404f4b0 con 0x7f65240c61c0 2024-04-01T00:13:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c487a640 1 -- 172.21.15.64:0/1946948249 learned_addr learned my addr 172.21.15.64:0/1946948249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8053910 con 0x7f11b8050190 2024-04-01T00:13:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 shutdown_connections 2024-04-01T00:13:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8053790 con 0x7f11b80392c0 2024-04-01T00:13:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/3440659789 >> v2:172.21.15.64:3300/0 conn(0x7fb2a0162940 0x7fb2a00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f11c607d640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f11b8053a90 con 0x7f11b800aa90 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01624e0 0x7fb2a00cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/3440659789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a00cecb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 >> 172.21.15.64:0/3440659789 conn(0x7fb2a006b0c0 msgr2=0x7fb2a006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f1a91d35640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a8c0caf70 0x7f1a8c0c8ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c007fd0 tx=0x7f1a7c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 shutdown_connections 2024-04-01T00:13:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c507b640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b800aa90 0x7f11b804e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/3440659789 wait complete. 2024-04-01T00:13:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 -- 172.21.15.64:0/2127013792 shutdown_connections 2024-04-01T00:13:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c017070 con 0x7f1a8c0caf70 2024-04-01T00:13:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7fd32a981640 1 -- 172.21.15.64:0/2127013792 wait complete. 2024-04-01T00:13:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c487a640 1 -- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b800aa90 msgr2=0x7f11b804e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f1a9479a640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c167e40 con 0x7f1a8c0caf70 2024-04-01T00:13:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f1a9479a640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c168400 con 0x7f1a8c0caf70 2024-04-01T00:13:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fedac0057e0 con 0x7fedac009190 2024-04-01T00:13:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f65100399b0 con 0x7f65240c61c0 2024-04-01T00:13:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7c005ce0 con 0x7f1a8c0caf70 2024-04-01T00:13:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.307+0000 7f42db59e640 1 -- 172.21.15.64:0/2783638147 wait complete. 2024-04-01T00:13:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c007310 con 0x7f1a8c0caf70 2024-04-01T00:13:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f42db59e640 1 Processor -- start 2024-04-01T00:13:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c487a640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b800aa90 0x7f11b804e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f42db59e640 1 -- start start 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c487a640 1 -- 172.21.15.64:0/1946948249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8050190 msgr2=0x7f11b8189d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c487a640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8050190 0x7f11b8189d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.308+0000 7f11c487a640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b80e0410 con 0x7f11b80392c0 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f11c487a640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 0x7f11b81897c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e870 tx=0x7f11bc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f11c507b640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11b800aa90 0x7f11b804e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7fb2a4ebe640 1 Processor -- start 2024-04-01T00:13:56.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc017070 con 0x7f11b80392c0 2024-04-01T00:13:56.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7fb2a4ebe640 1 -- start start 2024-04-01T00:13:56.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f11c607d640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b80e06b0 con 0x7f11b80392c0 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7fd32a981640 1 Processor -- start 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f650806bb90 0x7f650806e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65100ad240 con 0x7f65240c61c0 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f6508073d30 0x7f6508076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6510051e50 con 0x7f6508073d30 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f65100ad6d0 con 0x7f65240c61c0 2024-04-01T00:13:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f6523fff640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f6508073d30 0x7f6508076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.309+0000 7fd32a981640 1 -- start start 2024-04-01T00:13:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7fd32a981640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd31c0dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f6523fff640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f6508073d30 0x7f6508076150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f11c607d640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b8187070 con 0x7f11b80392c0 2024-04-01T00:13:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7fd32a981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd31c036410 0x7fd31c0372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.310+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1a7c01a070 con 0x7f1a8c0caf70 2024-04-01T00:13:56.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd31c0dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c40051c0 0x7f42d404cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd32a981640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd31c034db0 0x7fd31c037820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d4033c50 0x7f42d404db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f1a9479a640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a74001030 con 0x7f1a8c0caf70 2024-04-01T00:13:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd32a981640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0501a0 con 0x7fd31c036410 2024-04-01T00:13:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f6522ffd640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f650806bb90 0x7f650806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd32a981640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c050020 con 0x7fd3140039b0 2024-04-01T00:13:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd32a981640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c050320 con 0x7fd31c034db0 2024-04-01T00:13:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd31c0dfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45006/0 (socket says 172.21.15.64:45006) 2024-04-01T00:13:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 -- 172.21.15.64:0/83676447 learned_addr learned my addr 172.21.15.64:0/83676447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 0x7f42d4038930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc00c940 con 0x7f11b80392c0 2024-04-01T00:13:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f1a82ffd640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a7006ba90 0x7f1a7006df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00cd30 con 0x7f11b80392c0 2024-04-01T00:13:56.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd328f1e640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd31c034db0 0x7fd31c037820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4054940 con 0x7f42d4033c50 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40547c0 con 0x7f42c40051c0 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42db59e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f42d404f550 con 0x7f42d417d4a0 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 0x7f42d4038930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fb2a4ebe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 0x7f42d4038930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38354/0 (socket says 172.21.15.64:38354) 2024-04-01T00:13:56.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42dad9d640 1 -- 172.21.15.64:0/1297596458 learned_addr learned my addr 172.21.15.64:0/1297596458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f42da59c640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c40051c0 0x7f42d404cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fb2a4ebe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2a01624e0 0x7fb2a00c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 -- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd31c034db0 msgr2=0x7fd31c037820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a015b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45014/0 (socket says 172.21.15.64:45014) 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7c089a90 con 0x7f1a8c0caf70 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f1a92536640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a7006ba90 0x7f1a7006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7f1a82ffd640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a70073c30 0x7f1a70076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a7c005ce0 con 0x7f1a70073c30 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1a7c090050 con 0x7f1a8c0caf70 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd31c034db0 0x7fd31c037820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 -- 172.21.15.64:0/83676447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd31c036410 msgr2=0x7fd31c0372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd31c036410 0x7fd31c0372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.311+0000 7fd323fff640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c035370 con 0x7fd3140039b0 2024-04-01T00:13:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f6522ffd640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f650806bb90 0x7f650806e050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65240b42c0 tx=0x7f6518008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f4154031af0 con 0x7f416815a050 2024-04-01T00:13:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fd323fff640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd31c0dfc50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd318009db0 tx=0x7fd318004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f1a92d37640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a70073c30 0x7f1a70076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415006b990 0x7f415006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f1a92536640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a7006ba90 0x7f1a7006df50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009560 tx=0x7f1a8803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1198001030 con 0x7f11b80392c0 2024-04-01T00:13:56.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318044020 con 0x7fd3140039b0 2024-04-01T00:13:56.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318035a00 con 0x7fd3140039b0 2024-04-01T00:13:56.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb2a4ebe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a00cb7d0 0x7fb2a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb2a4ebe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a006ed70 con 0x7fb2a01624e0 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb29effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2a01624e0 0x7fb2a00c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb29effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2a01624e0 0x7fb2a00c3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51210/0 (socket says 172.21.15.64:51210) 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb29effd640 1 -- 172.21.15.64:0/255940325 learned_addr learned my addr 172.21.15.64:0/255940325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a006ebf0 con 0x7fb2a015e260 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a00b1bd0 con 0x7fb2a00cb7d0 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41540acc80 con 0x7f416815a050 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f4166ffd640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415006b990 0x7f415006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7f4150073b30 0x7f4150075f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4150076600 con 0x7f4150073b30 2024-04-01T00:13:56.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41540ad070 con 0x7f416815a050 2024-04-01T00:13:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 -- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c40051c0 msgr2=0x7f42d404cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f1a92d37640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a70073c30 0x7f1a70076050 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318039040 con 0x7fd3140039b0 2024-04-01T00:13:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c40051c0 0x7f42d404cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 -- 172.21.15.64:0/1297596458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d4033c50 msgr2=0x7f42d404db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f4167fff640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7f4150073b30 0x7f4150075f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29ffff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a00cb7d0 0x7fb2a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d4033c50 0x7f42d404db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d404e330 con 0x7f42d417d4a0 2024-04-01T00:13:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42da59c640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c40051c0 0x7f42d404cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29f7fe640 1 -- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a00cb7d0 msgr2=0x7fb2a006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fd32a981640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c035650 con 0x7fd3140039b0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.312+0000 7f42dad9d640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 0x7f42d4038930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42dc06a030 tx=0x7f42dc069610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0b4040 con 0x7f42d417d4a0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc0afbb0 con 0x7f42d417d4a0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0ba470 con 0x7f42d417d4a0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fd32a981640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c035c10 con 0x7fd3140039b0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f42db59e640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d404e610 con 0x7f42d417d4a0 2024-04-01T00:13:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f42db59e640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d417cf80 con 0x7f42d417d4a0 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f11bc01a070 con 0x7f11b80392c0 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a00cb7d0 0x7fb2a006d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29f7fe640 1 -- 172.21.15.64:0/255940325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2a01624e0 msgr2=0x7fb2a00c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2a01624e0 0x7fb2a00c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29f7fe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a00c44d0 con 0x7fb2a015e260 2024-04-01T00:13:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fb29ffff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a00cb7d0 0x7fb2a006d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:56.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.313+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fedb4071b10 con 0x7fedac009190 2024-04-01T00:13:56.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f4166ffd640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415006b990 0x7f415006de50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41600097f0 tx=0x7f416000c060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11b5ffb640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f11a406ba60 0x7f11a406df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb29f7fe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a015b940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb290033000 tx=0x7fb290004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f4167fff640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7f4150073b30 0x7f4150075f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f11bc08a620 con 0x7f11b80392c0 2024-04-01T00:13:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== osd.0 v2:172.21.15.64:6800/3875769055 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a7c005ce0 con 0x7f1a70073c30 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290048070 con 0x7fb2a015e260 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290030670 con 0x7fb2a015e260 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290034040 con 0x7fb2a015e260 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11b5ffb640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7f11a4073ce0 0x7f11a4076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11bc015e50 con 0x7f11a4073ce0 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f11bc091050 con 0x7f11b80392c0 2024-04-01T00:13:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fedb17fa640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fed9806ba60 0x7fed9806df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f11c507b640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f11a406ba60 0x7f11a406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fedb4105d90 con 0x7fedac009190 2024-04-01T00:13:56.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a00c47b0 con 0x7fb2a015e260 2024-04-01T00:13:56.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a00c4d70 con 0x7fb2a015e260 2024-04-01T00:13:56.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fedb17fa640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7fed98073c80 0x7fed980760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed98076750 con 0x7fed98073c80 2024-04-01T00:13:56.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fedb4106180 con 0x7fedac009190 2024-04-01T00:13:56.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7fedb3fff640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fed9806ba60 0x7fed9806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd31c035ff0 con 0x7fd3140039b0 2024-04-01T00:13:56.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.314+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== osd.2 v2:172.21.15.64:6824/2000381811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6510051e50 con 0x7f6508073d30 2024-04-01T00:13:56.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== osd.3 v2:172.21.15.64:6808/2132154805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4150076600 con 0x7f4150073b30 2024-04-01T00:13:56.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7fedb924b640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7fed98073c80 0x7fed980760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7f11c587c640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7f11a4073ce0 0x7f11a4076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f42dc0c4060 con 0x7f42d417d4a0 2024-04-01T00:13:56.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fedb3fff640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fed9806ba60 0x7fed9806df20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedac0eada0 tx=0x7feda8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fedb924b640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7fed98073c80 0x7fed980760a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.315+0000 7f42db59e640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42d4005390 con 0x7f42d417d4a0 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42caffd640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f42bc06ba60 0x7f42bc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42dc126d80 con 0x7f42d417d4a0 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42da59c640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f42bc06ba60 0x7f42bc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f11c507b640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f11a406ba60 0x7f11a406df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11b0005880 tx=0x7f11b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f11c587c640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7f11a4073ce0 0x7f11a4076100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd318047020 con 0x7fd3140039b0 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd30806b990 0x7fd30806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd3180acd30 con 0x7fd3140039b0 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7fd308073b30 0x7fd308075f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd31802c910 con 0x7fd308073b30 2024-04-01T00:13:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3180b2300 con 0x7fd3140039b0 2024-04-01T00:13:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42caffd640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f42bc073c80 0x7f42bc0760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42dc0c0ec0 con 0x7f42bc073c80 2024-04-01T00:13:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f42dc12c370 con 0x7f42d417d4a0 2024-04-01T00:13:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7f42d9d9b640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f42bc073c80 0x7f42bc0760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.316+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2a015e640 con 0x7fb2a015e260 2024-04-01T00:13:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb29004b070 con 0x7fb2a015e260 2024-04-01T00:13:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7f42da59c640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f42bc06ba60 0x7f42bc06df20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006fd0 tx=0x7f42d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7f42d9d9b640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f42bc073c80 0x7f42bc0760a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fd328f1e640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7fd308073b30 0x7fd308075f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fb287fff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb28c06b9e0 0x7fb28c06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fd3237fe640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd30806b990 0x7fd30806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== osd.1 v2:172.21.15.64:6816/199266641 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed98076750 con 0x7fed98073c80 2024-04-01T00:13:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fb29effd640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb28c06b9e0 0x7fb28c06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.317+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2900ac8b0 con 0x7fb2a015e260 2024-04-01T00:13:56.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fd328f1e640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7fd308073b30 0x7fd308075f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== osd.4 v2:172.21.15.100:6816/3826994341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11bc015e50 con 0x7f11a4073ce0 2024-04-01T00:13:56.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fb287fff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fb28c073c00 0x7fb28c076020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb28c0766d0 con 0x7fb28c073c00 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb2900b12a0 con 0x7fb2a015e260 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fb29ffff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fb28c073c00 0x7fb28c076020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fd3237fe640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd30806b990 0x7fd30806de50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0e7710 tx=0x7fd310008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== osd.6 v2:172.21.15.100:6808/3251071123 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42dc0c0ec0 con 0x7f42bc073c80 2024-04-01T00:13:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fb29effd640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb28c06b9e0 0x7fb28c06dea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00ba8e0 tx=0x7fb294008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.318+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== osd.5 v2:172.21.15.100:6800/64687588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd31802c910 con 0x7fd308073b30 2024-04-01T00:13:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.319+0000 7fb29ffff640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fb28c073c00 0x7fb28c076020 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.319+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== osd.7 v2:172.21.15.100:6824/2776403688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb28c0766d0 con 0x7fb28c073c00 2024-04-01T00:13:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.346+0000 7f1a9479a640 1 -- 172.21.15.64:0/1593583224 --> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a74002dd0 con 0x7f1a70073c30 2024-04-01T00:13:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a82ffd640 1 -- 172.21.15.64:0/1593583224 <== osd.0 v2:172.21.15.64:6800/3875769055 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a74002dd0 con 0x7f1a70073c30 2024-04-01T00:13:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a70073c30 msgr2=0x7f1a70076050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a70073c30 0x7f1a70076050 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a7006ba90 msgr2=0x7f1a7006df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 --> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65240ca760 con 0x7f6508073d30 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a7006ba90 0x7f1a7006df50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009560 tx=0x7f1a8803c040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a8c0caf70 msgr2=0x7f1a8c0c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.347+0000 7f1a80ff9640 1 --2- 172.21.15.64:0/1593583224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a8c0caf70 0x7f1a8c0c8ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c007fd0 tx=0x7f1a7c00c890 comp rx=0 tx=0).stop 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f1a92d37640 1 -- 172.21.15.64:0/1593583224 reap_dead start 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 shutdown_connections 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 >> 172.21.15.64:0/1593583224 conn(0x7f1a8c0cfa90 msgr2=0x7f1a8c072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f650ffff640 1 -- 172.21.15.64:0/3446560292 <== osd.2 v2:172.21.15.64:6824/2000381811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65240ca760 con 0x7f6508073d30 2024-04-01T00:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f6508073d30 msgr2=0x7f6508076150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.348+0000 7f6529b7a640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f6508073d30 0x7f6508076150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 shutdown_connections 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f650806bb90 msgr2=0x7f650806e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f650806bb90 0x7f650806e050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65240b42c0 tx=0x7f6518008040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 msgr2=0x7f652415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 --2- 172.21.15.64:0/3446560292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65240c61c0 0x7f652415c240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f65100046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6523fff640 1 -- 172.21.15.64:0/3446560292 reap_dead start 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 shutdown_connections 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 >> 172.21.15.64:0/3446560292 conn(0x7f65240cf990 msgr2=0x7f65240b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 shutdown_connections 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f6529b7a640 1 -- 172.21.15.64:0/3446560292 wait complete. 2024-04-01T00:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.349+0000 7f1a80ff9640 1 -- 172.21.15.64:0/1593583224 wait complete. 2024-04-01T00:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.350+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 --> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41680681a0 con 0x7f4150073b30 2024-04-01T00:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.350+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 --> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1198002dd0 con 0x7f11a4073ce0 2024-04-01T00:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.350+0000 7f415f7fe640 1 -- 172.21.15.64:0/1779886327 <== osd.3 v2:172.21.15.64:6808/2132154805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41680681a0 con 0x7f4150073b30 2024-04-01T00:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.350+0000 7f11b5ffb640 1 -- 172.21.15.64:0/1946948249 <== osd.4 v2:172.21.15.100:6816/3826994341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1198002dd0 con 0x7f11a4073ce0 2024-04-01T00:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 --> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd31c005810 con 0x7fd308073b30 2024-04-01T00:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7f11a4073ce0 msgr2=0x7f11a4076100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7f11a4073ce0 0x7f11a4076100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.350+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 --> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fedac0057e0 con 0x7fed98073c80 2024-04-01T00:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7fd320ff9640 1 -- 172.21.15.64:0/83676447 <== osd.5 v2:172.21.15.100:6800/64687588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd31c005810 con 0x7fd308073b30 2024-04-01T00:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f11a406ba60 msgr2=0x7f11a406df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7fedb17fa640 1 -- 172.21.15.64:0/2582559424 <== osd.1 v2:172.21.15.64:6816/199266641 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fedac0057e0 con 0x7fed98073c80 2024-04-01T00:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f11a406ba60 0x7f11a406df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11b0005880 tx=0x7f11b0009040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f42db59e640 1 -- 172.21.15.64:0/1297596458 --> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42d4047610 con 0x7f42bc073c80 2024-04-01T00:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 msgr2=0x7f11b81897c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 --2- 172.21.15.64:0/1946948249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11b80392c0 0x7f11b81897c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e870 tx=0x7f11bc00ed30 comp rx=0 tx=0).stop 2024-04-01T00:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11c587c640 1 -- 172.21.15.64:0/1946948249 reap_dead start 2024-04-01T00:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7f4150073b30 msgr2=0x7f4150075f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f416d72c640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7f4150073b30 0x7f4150075f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 shutdown_connections 2024-04-01T00:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 >> 172.21.15.64:0/1946948249 conn(0x7f11b8009fa0 msgr2=0x7f11b8050570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f42caffd640 1 -- 172.21.15.64:0/1297596458 <== osd.6 v2:172.21.15.100:6808/3251071123 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42d4047610 con 0x7f42bc073c80 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 shutdown_connections 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.351+0000 7f11ab7fe640 1 -- 172.21.15.64:0/1946948249 wait complete. 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7fed98073c80 msgr2=0x7fed980760a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7fed98073c80 0x7fed980760a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415006b990 msgr2=0x7f415006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415006b990 0x7f415006de50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41600097f0 tx=0x7f416000c060 comp rx=0 tx=0).stop 2024-04-01T00:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416815a050 msgr2=0x7f41681f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 --2- 172.21.15.64:0/1779886327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416815a050 0x7f41681f85b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4168074530 tx=0x7f41540046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f4167fff640 1 -- 172.21.15.64:0/1779886327 reap_dead start 2024-04-01T00:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 shutdown_connections 2024-04-01T00:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 >> 172.21.15.64:0/1779886327 conn(0x7f416809a980 msgr2=0x7f4168074fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fed9806ba60 msgr2=0x7fed9806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fed9806ba60 0x7fed9806df20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedac0eada0 tx=0x7feda8009040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 msgr2=0x7fedac0e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fed9effd640 1 --2- 172.21.15.64:0/2582559424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedac009190 0x7fedac0e5450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fedb4050040 tx=0x7fedb402d3d0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7fd308073b30 msgr2=0x7fd308075f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7fd308073b30 0x7fd308075f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd30806b990 msgr2=0x7fd30806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd30806b990 0x7fd30806de50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0e7710 tx=0x7fd310008040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 msgr2=0x7fd31c0dfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd30e7fc640 1 --2- 172.21.15.64:0/83676447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3140039b0 0x7fd31c0dfc50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd318009db0 tx=0x7fd318004730 comp rx=0 tx=0).stop 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fd328f1e640 1 -- 172.21.15.64:0/83676447 reap_dead start 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f42bc073c80 msgr2=0x7f42bc0760a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f42bc073c80 0x7f42bc0760a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f42bc06ba60 msgr2=0x7f42bc06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f42bc06ba60 0x7f42bc06df20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006fd0 tx=0x7f42d0008040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 msgr2=0x7f42d4038930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 --2- 172.21.15.64:0/1297596458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d417d4a0 0x7f42d4038930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42dc06a030 tx=0x7f42dc069610 comp rx=0 tx=0).stop 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42dad9d640 1 -- 172.21.15.64:0/1297596458 reap_dead start 2024-04-01T00:13:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 shutdown_connections 2024-04-01T00:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 >> 172.21.15.64:0/1297596458 conn(0x7f42d4009fb0 msgr2=0x7f42d40e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7fedb924b640 1 -- 172.21.15.64:0/2582559424 reap_dead start 2024-04-01T00:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 shutdown_connections 2024-04-01T00:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.352+0000 7f42c8ff9640 1 -- 172.21.15.64:0/1297596458 wait complete. 2024-04-01T00:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 shutdown_connections 2024-04-01T00:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 shutdown_connections 2024-04-01T00:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 shutdown_connections 2024-04-01T00:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 >> 172.21.15.64:0/2582559424 conn(0x7fedac01ef40 msgr2=0x7fedac054310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 >> 172.21.15.64:0/83676447 conn(0x7fd31c055380 msgr2=0x7fd31c04a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 shutdown_connections 2024-04-01T00:13:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7f416d72c640 1 -- 172.21.15.64:0/1779886327 wait complete. 2024-04-01T00:13:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fed9effd640 1 -- 172.21.15.64:0/2582559424 wait complete. 2024-04-01T00:13:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 shutdown_connections 2024-04-01T00:13:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 --> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2a004f4b0 con 0x7fb28c073c00 2024-04-01T00:13:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.353+0000 7fd30e7fc640 1 -- 172.21.15.64:0/83676447 wait complete. 2024-04-01T00:13:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.354+0000 7fb287fff640 1 -- 172.21.15.64:0/255940325 <== osd.7 v2:172.21.15.100:6824/2776403688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb2a004f4b0 con 0x7fb28c073c00 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.354+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fb28c073c00 msgr2=0x7fb28c076020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.354+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fb28c073c00 0x7fb28c076020 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb28c06b9e0 msgr2=0x7fb28c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb28c06b9e0 0x7fb28c06dea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00ba8e0 tx=0x7fb294008040 comp rx=0 tx=0).stop 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 msgr2=0x7fb2a015b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 --2- 172.21.15.64:0/255940325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2a015e260 0x7fb2a015b940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb290033000 tx=0x7fb290004730 comp rx=0 tx=0).stop 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb29ffff640 1 -- 172.21.15.64:0/255940325 reap_dead start 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 shutdown_connections 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.356+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 >> 172.21.15.64:0/255940325 conn(0x7fb2a006b0c0 msgr2=0x7fb2a00b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.357+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 shutdown_connections 2024-04-01T00:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.357+0000 7fb2a4ebe640 1 -- 172.21.15.64:0/255940325 wait complete. 2024-04-01T00:13:56.418 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:56.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-01T00:13:56.419 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:56.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-01T00:13:56.420 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:56.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-01T00:13:56.421 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:56.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-01T00:13:56.422 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:56.422 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:56.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-01T00:13:56.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-01T00:13:56.424 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:56.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-01T00:13:56.425 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:56.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-01T00:13:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7f936ffff640 1 Processor -- start 2024-04-01T00:13:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7f936ffff640 1 -- start start 2024-04-01T00:13:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7f936ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f936804fa80 0x7f936804fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7f936ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9368054260 0x7f93680389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7f936ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93680503a0 con 0x7f93680df410 2024-04-01T00:13:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7f936ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f9368006890 con 0x7f936804fa80 2024-04-01T00:13:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7f936ffff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9368006ae0 con 0x7f9368054260 2024-04-01T00:13:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936e7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f93680df410 0x7f93680382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48016/0 (socket says 172.21.15.64:48016) 2024-04-01T00:13:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936e7fc640 1 -- 172.21.15.64:0/3697011045 learned_addr learned my addr 172.21.15.64:0/3697011045 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7fcf29e3b640 1 Processor -- start 2024-04-01T00:13:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936effd640 1 --2- 172.21.15.64:0/3697011045 >> v2:172.21.15.64:3300/0 conn(0x7f936804fa80 0x7f936804fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.629+0000 7fde4e9c9640 1 Processor -- start 2024-04-01T00:13:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936effd640 1 -- 172.21.15.64:0/3697011045 >> v2:172.21.15.100:3300/0 conn(0x7f9368054260 msgr2=0x7f93680389c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-01T00:13:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936effd640 1 --2- 172.21.15.64:0/3697011045 >> v2:172.21.15.100:3300/0 conn(0x7f9368054260 0x7f93680389c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936effd640 1 -- 172.21.15.64:0/3697011045 >> v1:172.21.15.64:6789/0 conn(0x7f93680df410 legacy=0x7f93680382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7f936effd640 1 -- 172.21.15.64:0/3697011045 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9368006dc0 con 0x7f936804fa80 2024-04-01T00:13:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.629+0000 7fde4e9c9640 1 -- start start 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4e9c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4e9c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 0x7fde480ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4e9c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde480ca850 con 0x7fde48162740 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4e9c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fde480ca620 con 0x7fde48071590 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4e9c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fde48071b50 con 0x7fde480caaf0 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde477fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde477fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38378/0 (socket says 172.21.15.64:38378) 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde477fe640 1 -- 172.21.15.64:0/2741024405 learned_addr learned my addr 172.21.15.64:0/2741024405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.630+0000 7fde4cf66640 1 --2- 172.21.15.64:0/2741024405 >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 0x7fde480ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7fde477fe640 1 -- 172.21.15.64:0/2741024405 >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 msgr2=0x7fde480ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7fde477fe640 1 --2- 172.21.15.64:0/2741024405 >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 0x7fde480ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7fde477fe640 1 -- 172.21.15.64:0/2741024405 >> v1:172.21.15.64:6789/0 conn(0x7fde48162740 legacy=0x7fde48070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.631+0000 7fde477fe640 1 -- 172.21.15.64:0/2741024405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde48071e30 con 0x7fde480caaf0 2024-04-01T00:13:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7fde4cf66640 1 --2- 172.21.15.64:0/2741024405 >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 0x7fde480ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.632+0000 7fcf29e3b640 1 -- start start 2024-04-01T00:13:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.633+0000 7fde477fe640 1 --2- 172.21.15.64:0/2741024405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde38009870 tx=0x7fde3802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50de7421b5bc0fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.633+0000 7fe8d0684640 1 Processor -- start 2024-04-01T00:13:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.633+0000 7fde46ffd640 1 -- 172.21.15.64:0/2741024405 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38035020 con 0x7fde480caaf0 2024-04-01T00:13:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.633+0000 7fde46ffd640 1 -- 172.21.15.64:0/2741024405 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde38031870 con 0x7fde480caaf0 2024-04-01T00:13:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.633+0000 7fde46ffd640 1 -- 172.21.15.64:0/2741024405 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38031c80 con 0x7fde480caaf0 2024-04-01T00:13:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf29e3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 0x7fcf240c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf29e3b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf240caf70 0x7fcf24071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf29e3b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2406f110 con 0x7fcf240c5880 2024-04-01T00:13:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf29e3b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcf24071260 con 0x7fcf240caf70 2024-04-01T00:13:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf29e3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf240c63c0 con 0x7fcf24159fc0 2024-04-01T00:13:56.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7f936effd640 1 --2- 172.21.15.64:0/3697011045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f936804fe60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f936800dc50 tx=0x7f936002df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f6109051d7d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fd294b77640 1 Processor -- start 2024-04-01T00:13:56.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fd294b77640 1 -- start start 2024-04-01T00:13:56.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7f936dffb640 1 -- 172.21.15.64:0/3697011045 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360034070 con 0x7f936804fa80 2024-04-01T00:13:56.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7f936dffb640 1 -- 172.21.15.64:0/3697011045 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936002eca0 con 0x7f936804fa80 2024-04-01T00:13:56.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf22ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcf240c5880 0x7fcf240ca860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48026/0 (socket says 172.21.15.64:48026) 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf237fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 0x7fcf240c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fde4e9c9640 1 -- 172.21.15.64:0/2741024405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 msgr2=0x7fde480c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fde4e9c9640 1 --2- 172.21.15.64:0/2741024405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde38009870 tx=0x7fde3802b910 comp rx=0 tx=0).stop 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 -- 172.21.15.64:0/2741024405 shutdown_connections 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 --2- 172.21.15.64:0/2741024405 >> v2:172.21.15.64:3300/0 conn(0x7fde48071590 0x7fde480ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 --2- 172.21.15.64:0/2741024405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde480c5e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fcf22ffd640 1 -- 172.21.15.64:0/1727299884 learned_addr learned my addr 172.21.15.64:0/1727299884 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 -- 172.21.15.64:0/2741024405 >> 172.21.15.64:0/2741024405 conn(0x7fde480cfa90 msgr2=0x7fde480cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.634+0000 7fe8d0684640 1 -- start start 2024-04-01T00:13:56.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf23fff640 1 --2- 172.21.15.64:0/1727299884 >> v2:172.21.15.100:3300/0 conn(0x7fcf240caf70 0x7fcf24071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7ff76f331640 1 Processor -- start 2024-04-01T00:13:56.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936dffb640 1 -- 172.21.15.64:0/3697011045 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360038620 con 0x7f936804fa80 2024-04-01T00:13:56.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7ff76f331640 1 -- start start 2024-04-01T00:13:56.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 -- 172.21.15.64:0/2741024405 shutdown_connections 2024-04-01T00:13:56.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f195d0e9640 1 Processor -- start 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf237fe640 1 -- 172.21.15.64:0/1727299884 >> v2:172.21.15.100:3300/0 conn(0x7fcf240caf70 msgr2=0x7fcf24071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf237fe640 1 --2- 172.21.15.64:0/1727299884 >> v2:172.21.15.100:3300/0 conn(0x7fcf240caf70 0x7fcf24071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf237fe640 1 -- 172.21.15.64:0/1727299884 >> v1:172.21.15.64:6789/0 conn(0x7fcf240c5880 legacy=0x7fcf240ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf227fc640 1 -- 172.21.15.64:0/1727299884 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018544408 0 0) 0x7fcf2406f110 con 0x7fcf240c5880 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 -- 172.21.15.64:0/3697011045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 msgr2=0x7f936804fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 --2- 172.21.15.64:0/3697011045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f936804fe60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f936800dc50 tx=0x7f936002df70 comp rx=0 tx=0).stop 2024-04-01T00:13:56.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fcf237fe640 1 -- 172.21.15.64:0/1727299884 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2415a3a0 con 0x7fcf24159fc0 2024-04-01T00:13:56.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fe8d0684640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f5dfeeca640 1 Processor -- start 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 -- 172.21.15.64:0/3697011045 shutdown_connections 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 --2- 172.21.15.64:0/3697011045 >> v2:172.21.15.100:3300/0 conn(0x7f9368054260 0x7f93680389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 --2- 172.21.15.64:0/3697011045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f936804fe60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 -- 172.21.15.64:0/3697011045 >> 172.21.15.64:0/3697011045 conn(0x7f9368009fa0 msgr2=0x7f936800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fe8d0684640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8c81666b0 0x7fe8c8162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f195d0e9640 1 -- start start 2024-04-01T00:13:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f5dfeeca640 1 -- start start 2024-04-01T00:13:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fe8d0684640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c80cde60 con 0x7fe8c80ceec0 2024-04-01T00:13:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7f936ffff640 1 -- 172.21.15.64:0/3697011045 shutdown_connections 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fe8d0684640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c80ce100 con 0x7fe8c81666b0 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd294b77640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2900c5c50 0x7fd2900cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd294b77640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd29006dbf0 con 0x7fd2900cd660 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd294b77640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2900cb3b0 con 0x7fd2900c6500 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd294b77640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd2900cb690 con 0x7fd2900c5c50 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd28e59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd2900cd660 0x7fd29006d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48036/0 (socket says 172.21.15.64:48036) 2024-04-01T00:13:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd28e59c640 1 -- 172.21.15.64:0/1699005580 learned_addr learned my addr 172.21.15.64:0/1699005580 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fe8d0684640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c80ce3e0 con 0x7fe8c8162280 2024-04-01T00:13:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.635+0000 7fde4e9c9640 1 -- 172.21.15.64:0/2741024405 wait complete. 2024-04-01T00:13:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7ff76f331640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff76815be70 0x7ff7680d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546899729 0 0) 0x7fd2900cb3b0 con 0x7fd2900c6500 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd278003440 con 0x7fd2900c6500 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7ff76f331640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 0x7ff7680b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7ff76f331640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7680cd5f0 con 0x7ff7680d1c70 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7ff76f331640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff7680b14c0 con 0x7ff76815be70 2024-04-01T00:13:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7ff76f331640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7680cd8d0 con 0x7ff7680ccfd0 2024-04-01T00:13:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fe8ce420640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fe8ce420640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38404/0 (socket says 172.21.15.64:38404) 2024-04-01T00:13:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7fe8ce420640 1 -- 172.21.15.64:0/3891741332 learned_addr learned my addr 172.21.15.64:0/3891741332 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2172269847 0 0) 0x7fd278003440 con 0x7fd2900c6500 2024-04-01T00:13:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76db2e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff7680d1c70 0x7ff7680cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:50374/0 (socket says 172.21.15.64:50374) 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fe8cec21640 1 --2- 172.21.15.64:0/3891741332 >> v2:172.21.15.100:3300/0 conn(0x7fe8c81666b0 0x7fe8c8162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 >> v2:172.21.15.64:3300/0 conn(0x7fd2900c5c50 msgr2=0x7fd2900cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 --2- 172.21.15.64:0/1699005580 >> v2:172.21.15.64:3300/0 conn(0x7fd2900c5c50 0x7fd2900cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 >> v1:172.21.15.64:6789/0 conn(0x7fd2900cd660 legacy=0x7fd29006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900c4a60 con 0x7fd2900c6500 2024-04-01T00:13:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fd284001670 con 0x7fd2900c6500 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 >> v1:172.21.15.100:6789/0 conn(0x7fd2900c6500 legacy=0x7fd2900c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfeeca640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df00df210 0x7f5df0038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfeeca640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5df0050190 0x7f5df0006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfeeca640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df00544b0 con 0x7f5df0053c60 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfeeca640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f5df00077d0 con 0x7f5df0050190 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfeeca640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5df00541d0 con 0x7f5df00df210 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfdec8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df00df210 0x7f5df0038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfdec8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df00df210 0x7f5df0038070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45062/0 (socket says 172.21.15.64:45062) 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f5dfdec8640 1 -- 172.21.15.64:0/3442212773 learned_addr learned my addr 172.21.15.64:0/3442212773 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f195d0e9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f1950038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f195d0e9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500544b0 con 0x7f19500df210 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f195d0e9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500541d0 con 0x7f1950053c60 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f1957fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f19500df210 0x7f195004fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48038/0 (socket says 172.21.15.64:48038) 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f1957fff640 1 -- 172.21.15.64:0/2893871591 learned_addr learned my addr 172.21.15.64:0/2893871591 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1950005f20 con 0x7f1950050190 2024-04-01T00:13:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd278003ac0 0x7fd278003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278008a50 0x7fd27800aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 0x7fd27800dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900cb690 con 0x7fd278008a50 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd27800e520 con 0x7fd27800b780 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd27800e720 con 0x7fd278003ac0 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28ed9d640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 0x7fd27800dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28dd9b640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278008a50 0x7fd27800aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fd28e59c640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd278003ac0 0x7fd278003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76db2e640 1 -- 172.21.15.64:0/3476742985 learned_addr learned my addr 172.21.15.64:0/3476742985 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76eb30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 0x7ff7680b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76e32f640 1 --2- 172.21.15.64:0/3476742985 >> v2:172.21.15.100:3300/0 conn(0x7ff76815be70 0x7ff7680d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76eb30640 1 -- 172.21.15.64:0/3476742985 >> v2:172.21.15.100:3300/0 conn(0x7ff76815be70 msgr2=0x7ff7680d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76eb30640 1 --2- 172.21.15.64:0/3476742985 >> v2:172.21.15.100:3300/0 conn(0x7ff76815be70 0x7ff7680d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76eb30640 1 -- 172.21.15.64:0/3476742985 >> v1:172.21.15.100:6789/0 conn(0x7ff7680d1c70 legacy=0x7ff7680cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7ff76eb30640 1 -- 172.21.15.64:0/3476742985 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7680b1710 con 0x7ff7680ccfd0 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.636+0000 7f936ffff640 1 -- 172.21.15.64:0/3697011045 wait complete. 2024-04-01T00:13:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f936ffff640 1 Processor -- start 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f5dfdec8640 1 -- 172.21.15.64:0/3442212773 >> v2:172.21.15.64:3300/0 conn(0x7f5df0050190 msgr2=0x7f5df0006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f5dfdec8640 1 --2- 172.21.15.64:0/3442212773 >> v2:172.21.15.64:3300/0 conn(0x7f5df0050190 0x7f5df0006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f5dfdec8640 1 -- 172.21.15.64:0/3442212773 >> v1:172.21.15.100:6789/0 conn(0x7f5df0053c60 legacy=0x7f5df004fa80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f5dfdec8640 1 -- 172.21.15.64:0/3442212773 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df0005f20 con 0x7f5df00df210 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fe8ce420640 1 -- 172.21.15.64:0/3891741332 >> v2:172.21.15.100:3300/0 conn(0x7fe8c81666b0 msgr2=0x7fe8c8162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fe8ce420640 1 --2- 172.21.15.64:0/3891741332 >> v2:172.21.15.100:3300/0 conn(0x7fe8c81666b0 0x7fe8c8162660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fe8ce420640 1 -- 172.21.15.64:0/3891741332 >> v1:172.21.15.100:6789/0 conn(0x7fe8c80ceec0 legacy=0x7fe8c8165fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fe8ce420640 1 -- 172.21.15.64:0/3891741332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c80ce6c0 con 0x7fe8c8162280 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28ed9d640 1 -- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd278003ac0 msgr2=0x7fd278003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28ed9d640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd278003ac0 0x7fd278003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28ed9d640 1 -- 172.21.15.64:0/1699005580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278008a50 msgr2=0x7fd27800aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28ed9d640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278008a50 0x7fd27800aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28ed9d640 1 -- 172.21.15.64:0/1699005580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900c4a60 con 0x7fd27800b780 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fcf237fe640 1 --2- 172.21.15.64:0/1727299884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 0x7fcf240c5340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf240c6540 tx=0x7fcf1402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534b9cbbc510de62 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fde4e9c9640 1 Processor -- start 2024-04-01T00:13:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.637+0000 7fde4e9c9640 1 -- start start 2024-04-01T00:13:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794120722 0 0) 0x7f19500544b0 con 0x7f19500df210 2024-04-01T00:13:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1948003520 con 0x7f19500df210 2024-04-01T00:13:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28dd9b640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278008a50 0x7fd27800aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fd28e59c640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd278003ac0 0x7fd278003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fcf227fc640 1 -- 172.21.15.64:0/1727299884 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14035020 con 0x7fcf24159fc0 2024-04-01T00:13:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7ff76eb30640 1 --2- 172.21.15.64:0/3476742985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 0x7ff7680b1f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76002bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101cfea17eae915d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fcf227fc640 1 -- 172.21.15.64:0/1727299884 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf14031810 con 0x7fcf24159fc0 2024-04-01T00:13:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fcf227fc640 1 -- 172.21.15.64:0/1727299884 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14031be0 con 0x7fcf24159fc0 2024-04-01T00:13:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f19577fe640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f1950038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33425500 0 0) 0x7f1948003520 con 0x7f19500df210 2024-04-01T00:13:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 msgr2=0x7f1950038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7ff76d32d640 1 -- 172.21.15.64:0/3476742985 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760039070 con 0x7ff7680ccfd0 2024-04-01T00:13:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f1950038070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7ff76d32d640 1 -- 172.21.15.64:0/3476742985 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76002c980 con 0x7ff7680ccfd0 2024-04-01T00:13:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7ff76d32d640 1 -- 172.21.15.64:0/3476742985 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760039070 con 0x7ff7680ccfd0 2024-04-01T00:13:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 >> v1:172.21.15.100:6789/0 conn(0x7f1950053c60 legacy=0x7f1950006a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T00:13:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1950006200 con 0x7f19500df210 2024-04-01T00:13:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f5dfdec8640 1 --2- 172.21.15.64:0/3442212773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df0038070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5df804f9d0 tx=0x7f5df80a91b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1532069a39d74f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7fde4e9c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde4e9c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde4814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde4e9c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48162740 0x7fde480c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45068/0 (socket says 172.21.15.64:45068) 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 -- 172.21.15.64:0/3044397978 learned_addr learned my addr 172.21.15.64:0/3044397978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f194c001670 con 0x7f19500df210 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 >> v1:172.21.15.64:6789/0 conn(0x7f19500df210 legacy=0x7f195004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480037a0 0x7f1948003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fd28ed9d640 1 -- 172.21.15.64:0/1699005580 reap_dead start 2024-04-01T00:13:56.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f5dfcec6640 1 -- 172.21.15.64:0/3442212773 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b6070 con 0x7f5df00df210 2024-04-01T00:13:56.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fd28ed9d640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 0x7fd27800dc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd280009860 tx=0x7fd278003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50850158d62447eb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f5dfcec6640 1 -- 172.21.15.64:0/3442212773 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80b1640 con 0x7f5df00df210 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde477fe640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde4814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde480cdf30 con 0x7fde48162740 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f5dfcec6640 1 -- 172.21.15.64:0/3442212773 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b1a70 con 0x7f5df00df210 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde480cddb0 con 0x7fde48071590 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fde480ce0b0 con 0x7fde480caaf0 2024-04-01T00:13:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.638+0000 7f936ffff640 1 -- start start 2024-04-01T00:13:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f936ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f9368180120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280009460 con 0x7fd27800b780 2024-04-01T00:13:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd280004220 con 0x7fd27800b780 2024-04-01T00:13:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1956ffd640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1948004200 0x7f194800b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1727299884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 msgr2=0x7fcf240c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fe8ce420640 1 --2- 172.21.15.64:0/3891741332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009e50 tx=0x7fe8c002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9097f89b9027cf4f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 -- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 msgr2=0x7fde4814fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde4814fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 -- 172.21.15.64:0/3044397978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48162740 msgr2=0x7fde480c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48162740 0x7fde480c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fde47fff640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4806dcf0 con 0x7fde48071590 2024-04-01T00:13:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7f1957fff640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480037a0 0x7f1948003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd28d59a640 1 -- 172.21.15.64:0/1699005580 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280009460 con 0x7fd27800b780 2024-04-01T00:13:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.639+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1727299884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 0x7fcf240c5340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf240c6540 tx=0x7fcf1402b8b0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 -- 172.21.15.64:0/3476742985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 msgr2=0x7ff7680b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 --2- 172.21.15.64:0/3476742985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 0x7ff7680b1f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76002bc50 comp rx=0 tx=0).stop 2024-04-01T00:13:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fde47fff640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c430 tx=0x7fde3c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f1956ffd640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f194800d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1950005f20 con 0x7f1948004200 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f194800dec0 con 0x7f19480037a0 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f194800e0c0 con 0x7f1950050190 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f19577fe640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1948004200 0x7f194800b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1727299884 shutdown_connections 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1727299884 >> v2:172.21.15.100:3300/0 conn(0x7fcf240caf70 0x7fcf24071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1727299884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf24159fc0 0x7fcf240c5340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1727299884 >> 172.21.15.64:0/1727299884 conn(0x7fcf240cfa90 msgr2=0x7fcf240cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1727299884 shutdown_connections 2024-04-01T00:13:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 -- 172.21.15.64:0/3476742985 shutdown_connections 2024-04-01T00:13:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 --2- 172.21.15.64:0/3476742985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680ccfd0 0x7ff7680b1f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 --2- 172.21.15.64:0/3476742985 >> v2:172.21.15.100:3300/0 conn(0x7ff76815be70 0x7ff7680d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 -- 172.21.15.64:0/3476742985 >> 172.21.15.64:0/3476742985 conn(0x7ff76809a980 msgr2=0x7ff76809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 -- 172.21.15.64:0/3476742985 shutdown_connections 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9368054260 0x7f93680485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9368052b30 con 0x7f936804fa80 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936ffff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9368007380 con 0x7f9368054260 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936ffff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9368052cb0 con 0x7f93680df410 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 -- 172.21.15.64:0/3442212773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 msgr2=0x7f5df0038070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/3442212773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df0038070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5df804f9d0 tx=0x7f5df80a91b0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38452/0 (socket says 172.21.15.64:38452) 2024-04-01T00:13:56.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1727299884 wait complete. 2024-04-01T00:13:56.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936f7fe640 1 -- 172.21.15.64:0/1655664606 learned_addr learned my addr 172.21.15.64:0/1655664606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f1957fff640 1 -- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 msgr2=0x7f194800d780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T00:13:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 -- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 msgr2=0x7fd27800dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 0x7fd27800dc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd280009860 tx=0x7fd278003730 comp rx=0 tx=0).stop 2024-04-01T00:13:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 -- 172.21.15.64:0/3442212773 shutdown_connections 2024-04-01T00:13:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/3442212773 >> v2:172.21.15.64:3300/0 conn(0x7f5df0050190 0x7f5df0006a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/3442212773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df0038070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f5dfeeca640 1 -- 172.21.15.64:0/3442212773 >> 172.21.15.64:0/3442212773 conn(0x7f5df0009fa0 msgr2=0x7f5df000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7ff76f331640 1 -- 172.21.15.64:0/3476742985 wait complete. 2024-04-01T00:13:56.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c017070 con 0x7fde48071590 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 -- 172.21.15.64:0/1699005580 shutdown_connections 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 --2- 172.21.15.64:0/1699005580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd27800b780 0x7fd27800dc40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 -- 172.21.15.64:0/1699005580 >> 172.21.15.64:0/1699005580 conn(0x7fd2900cfa90 msgr2=0x7fd2900cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 -- 172.21.15.64:0/1699005580 shutdown_connections 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fd294b77640 1 -- 172.21.15.64:0/1699005580 wait complete. 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4806cf30 con 0x7fde48071590 2024-04-01T00:13:56.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4806d4f0 con 0x7fde48071590 2024-04-01T00:13:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f5dfeeca640 1 -- 172.21.15.64:0/3442212773 shutdown_connections 2024-04-01T00:13:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f195c8e8640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f194800d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7f936effd640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f9368180120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 -- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9368054260 msgr2=0x7f93680485e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9368054260 0x7f93680485e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 -- 172.21.15.64:0/1655664606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 msgr2=0x7f9368180120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f9368180120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 -- 172.21.15.64:0/1655664606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936804d140 con 0x7f93680df410 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.640+0000 7fe8cd41e640 1 -- 172.21.15.64:0/3891741332 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c002cb00 con 0x7fe8c8162280 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7fe8cd41e640 1 -- 172.21.15.64:0/3891741332 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c002cca0 con 0x7fe8c8162280 2024-04-01T00:13:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7fe8cd41e640 1 -- 172.21.15.64:0/3891741332 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0034610 con 0x7fe8c8162280 2024-04-01T00:13:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f936f7fe640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f936400b410 tx=0x7f936400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c0052d0 con 0x7fde48071590 2024-04-01T00:13:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003810 con 0x7fde48071590 2024-04-01T00:13:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f1957fff640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f194800d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fcf29e3b640 1 Processor -- start 2024-04-01T00:13:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f5dfeeca640 1 -- 172.21.15.64:0/3442212773 wait complete. 2024-04-01T00:13:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f1957fff640 1 -- 172.21.15.64:0/2893871591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1948004200 msgr2=0x7f194800b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f1957fff640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1948004200 0x7f194800b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f1957fff640 1 -- 172.21.15.64:0/2893871591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1948003520 con 0x7f19480037a0 2024-04-01T00:13:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7ff76f331640 1 Processor -- start 2024-04-01T00:13:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f19577fe640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1948004200 0x7f194800b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f195c8e8640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950050190 0x7f194800d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:13:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7ff76f331640 1 -- start start 2024-04-01T00:13:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364015020 con 0x7f93680df410 2024-04-01T00:13:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.641+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9364011830 con 0x7f93680df410 2024-04-01T00:13:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fd294b77640 1 Processor -- start 2024-04-01T00:13:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364011c20 con 0x7f93680df410 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 -- 172.21.15.64:0/3891741332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 msgr2=0x7fe8c80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 --2- 172.21.15.64:0/3891741332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009e50 tx=0x7fe8c002bf60 comp rx=0 tx=0).stop 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 -- 172.21.15.64:0/3891741332 shutdown_connections 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 --2- 172.21.15.64:0/3891741332 >> v2:172.21.15.100:3300/0 conn(0x7fe8c81666b0 0x7fe8c8162660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 --2- 172.21.15.64:0/3891741332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c8162280 0x7fe8c80caaf0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009e50 tx=0x7fe8c002bf60 comp rx=0 tx=0).stop 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fe8d0684640 1 -- 172.21.15.64:0/3891741332 >> 172.21.15.64:0/3891741332 conn(0x7fe8c80cfa90 msgr2=0x7fe8c80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7f936ffff640 1 -- 172.21.15.64:0/1655664606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936804d3b0 con 0x7f93680df410 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f5dfeeca640 1 Processor -- start 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f5dfeeca640 1 -- start start 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fcf29e3b640 1 -- start start 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240c5880 0x7fcf241fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 0x7fcf24201570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24071490 con 0x7fcf240c5880 2024-04-01T00:13:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf240c60e0 con 0x7fcf24159fc0 2024-04-01T00:13:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fcf29e3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf24071770 con 0x7fcf240caf70 2024-04-01T00:13:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f1957fff640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480037a0 0x7f1948003ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f194c00c160 tx=0x7f1948004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6672ac794b69774e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f936ffff640 1 -- 172.21.15.64:0/1655664606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f936804d970 con 0x7f93680df410 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fe8d0684640 1 -- 172.21.15.64:0/3891741332 shutdown_connections 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c007c80 con 0x7f19480037a0 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7f1956ffd640 1 -- 172.21.15.64:0/2893871591 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c030070 con 0x7f19480037a0 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480037a0 msgr2=0x7f1948003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7f195d0e9640 1 --2- 172.21.15.64:0/2893871591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480037a0 0x7f1948003ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f194c00c160 tx=0x7f1948004820 comp rx=0 tx=0).stop 2024-04-01T00:13:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.642+0000 7fd294b77640 1 -- start start 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 0x7fd2901515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd29014ff80 0x7fd2900c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 0x7fd2901e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900cc0a0 con 0x7fd29014ff80 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900c50b0 con 0x7fd29014f6c0 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd294b77640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd2900cadd0 con 0x7fd27800b780 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd28ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 0x7fd2901e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd28ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 0x7fd2901e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45096/0 (socket says 172.21.15.64:45096) 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd28ed9d640 1 -- 172.21.15.64:0/3783528696 learned_addr learned my addr 172.21.15.64:0/3783528696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd28e59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 0x7fd2901515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fd28dd9b640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd29014ff80 0x7fd2900c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fcf237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240c5880 0x7fcf241fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fcf22ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fcf22ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38470/0 (socket says 172.21.15.64:38470) 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fcf22ffd640 1 -- 172.21.15.64:0/1980622097 learned_addr learned my addr 172.21.15.64:0/1980622097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fcf23fff640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 0x7fcf24201570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7ff76f331640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76f331640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680d1c70 0x7ff7680c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45084/0 (socket says 172.21.15.64:45084) 2024-04-01T00:13:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 -- 172.21.15.64:0/2273300498 learned_addr learned my addr 172.21.15.64:0/2273300498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76db2e640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680d1c70 0x7ff7680c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 0x7ff76815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff76815c250 con 0x7ff76815be70 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680d20d0 con 0x7ff7680ccfd0 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7680b2450 con 0x7ff7680d1c70 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76eb30640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 0x7ff76815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 -- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680d1c70 msgr2=0x7ff7680c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680d1c70 0x7ff7680c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 -- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 msgr2=0x7ff76815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 0x7ff76815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76e32f640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff768151920 con 0x7ff7680ccfd0 2024-04-01T00:13:56.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7ff76eb30640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 0x7ff76815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7f5dfeeca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7f5dfeeca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 0x7f5df0180570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7f5dfeeca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfeeca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0037900 con 0x7f5df0053c60 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfeeca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00064e0 con 0x7f5df00df210 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfeeca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5df0037a80 con 0x7f5df0050190 2024-04-01T00:13:56.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 -- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 msgr2=0x7fcf24201570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 0x7fcf24201570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fe8d0684640 1 -- 172.21.15.64:0/3891741332 wait complete. 2024-04-01T00:13:56.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 -- 172.21.15.64:0/1980622097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240c5880 msgr2=0x7fcf241fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240c5880 0x7fcf241fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 -- 172.21.15.64:0/1980622097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf241fe360 con 0x7fcf240caf70 2024-04-01T00:13:56.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7ff76e32f640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff75800d5f0 tx=0x7ff75800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f9364018020 con 0x7f93680df410 2024-04-01T00:13:56.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 -- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 msgr2=0x7fd2901e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 0x7fd2901e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf23fff640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 0x7fcf24201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:13:56.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfd6c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 0x7f5df0180570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfe6c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfd6c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 0x7f5df0180570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51304/0 (socket says 172.21.15.64:51304) 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfe6c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45102/0 (socket says 172.21.15.64:45102) 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.643+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde28005290 con 0x7fde48071590 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fde3c01a070 con 0x7fde48071590 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fde44ff9640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 0x7fde3406df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.644+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c0899d0 con 0x7fde48071590 2024-04-01T00:13:56.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 -- 172.21.15.64:0/3783528696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd29014ff80 msgr2=0x7fd2900c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfd6c7640 1 -- 172.21.15.64:0/155572176 learned_addr learned my addr 172.21.15.64:0/155572176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:38486/0 (socket says 172.21.15.64:38486) 2024-04-01T00:13:56.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c490 tx=0x7fcf1800c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd29014ff80 0x7fd2900c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 -- 172.21.15.64:0/3783528696 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290151b00 con 0x7fd27800b780 2024-04-01T00:13:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f195c8e8640 1 -- 172.21.15.64:0/2893871591 reap_dead start 2024-04-01T00:13:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fd28e59c640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 0x7fd2901515c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd28800dc60 tx=0x7fd28800b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 shutdown_connections 2024-04-01T00:13:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 >> 172.21.15.64:0/2893871591 conn(0x7f1950009fa0 msgr2=0x7f195000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:56.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f9357fff640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 0x7f935c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fde477fe640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 0x7fde3406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 -- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 msgr2=0x7f5df017d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 -- 172.21.15.64:0/155572176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 msgr2=0x7f5df0180570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 0x7f5df0180570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfdec8640 1 -- 172.21.15.64:0/155572176 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df017d170 con 0x7f5df0050190 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f5dfe6c9640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005ce0 con 0x7fcf240caf70 2024-04-01T00:13:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18007070 con 0x7fcf240caf70 2024-04-01T00:13:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18007460 con 0x7fcf240caf70 2024-04-01T00:13:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.645+0000 7f936ffff640 1 -- 172.21.15.64:0/1655664606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93680089c0 con 0x7f93680df410 2024-04-01T00:13:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f936effd640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 0x7f935c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 shutdown_connections 2024-04-01T00:13:56.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288012410 con 0x7fd27800b780 2024-04-01T00:13:56.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf24144b80 con 0x7fcf240caf70 2024-04-01T00:13:56.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2414f4d0 con 0x7fcf240caf70 2024-04-01T00:13:56.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f5dfdec8640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5df8050040 tx=0x7f5df8068f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f936408a690 con 0x7f93680df410 2024-04-01T00:13:56.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f936effd640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 0x7f935c06df20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9360039000 tx=0x7f9360034000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd290150540 con 0x7fd27800b780 2024-04-01T00:13:56.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fde477fe640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 0x7fde3406df20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde38036000 tx=0x7fde3803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288017070 con 0x7fd27800b780 2024-04-01T00:13:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7f195d0e9640 1 -- 172.21.15.64:0/2893871591 wait complete. 2024-04-01T00:13:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880127b0 con 0x7fd27800b780 2024-04-01T00:13:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fe8d0684640 1 Processor -- start 2024-04-01T00:13:56.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.646+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd290150b00 con 0x7fd27800b780 2024-04-01T00:13:56.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 -- start start 2024-04-01T00:13:56.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2414f2e0 con 0x7fcf240caf70 2024-04-01T00:13:56.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290151220 con 0x7fd27800b780 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c81666b0 0x7fe8c80b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c80c6b60 0x7fe8c8164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8166e10 con 0x7fe8c80ceec0 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8166c90 con 0x7fe8c80c6b60 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8d0684640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c80cb030 con 0x7fe8c81666b0 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8ce420640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8ce420640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51316/0 (socket says 172.21.15.64:51316) 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8ce420640 1 -- 172.21.15.64:0/388452122 learned_addr learned my addr 172.21.15.64:0/388452122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.647+0000 7fe8cec21640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c80c6b60 0x7fe8c8164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8cdc1f640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c81666b0 0x7fe8c80b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7f195d0e9640 1 Processor -- start 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 -- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c81666b0 msgr2=0x7fe8c80b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c81666b0 0x7fe8c80b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 -- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c80c6b60 msgr2=0x7fe8c8164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c80c6b60 0x7fe8c8164940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 -- 172.21.15.64:0/388452122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c80c6ff0 con 0x7fe8c80ceec0 2024-04-01T00:13:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7f195d0e9640 1 -- start start 2024-04-01T00:13:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8ce420640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002c4f0 tx=0x7fe8c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0043020 con 0x7fe8c80ceec0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c002ce50 con 0x7fe8c80ceec0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c80c72d0 con 0x7fe8c80ceec0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c003e690 con 0x7fe8c80ceec0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.648+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c80c7890 con 0x7fe8c80ceec0 2024-04-01T00:13:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950168d60 0x7f1950169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1950037a80 con 0x7f195004bb80 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1950037900 con 0x7f19500da9f0 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195d0e9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f19500df5f0 con 0x7f1950168d60 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195c8e8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195c8e8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51326/0 (socket says 172.21.15.64:51326) 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195c8e8640 1 -- 172.21.15.64:0/3954257314 learned_addr learned my addr 172.21.15.64:0/3954257314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45122/0 (socket says 172.21.15.64:45122) 2024-04-01T00:13:56.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f1957fff640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950168d60 0x7f1950169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c80681a0 con 0x7fe8c80ceec0 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 -- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950168d60 msgr2=0x7f1950169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950168d60 0x7f1950169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 -- 172.21.15.64:0/3954257314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 msgr2=0x7f195004c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f19577fe640 1 -- 172.21.15.64:0/3954257314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1950169680 con 0x7f19500da9f0 2024-04-01T00:13:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7f195c8e8640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.649+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580125f0 con 0x7ff7680ccfd0 2024-04-01T00:13:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f19577fe640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f194400f930 tx=0x7f194400fd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fde3c058000 con 0x7fde48071590 2024-04-01T00:13:56.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758019070 con 0x7ff7680ccfd0 2024-04-01T00:13:56.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194400db50 con 0x7f19500da9f0 2024-04-01T00:13:56.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580129d0 con 0x7ff7680ccfd0 2024-04-01T00:13:56.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194400dcf0 con 0x7f19500da9f0 2024-04-01T00:13:56.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194400dea0 con 0x7f19500da9f0 2024-04-01T00:13:56.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff768150c80 con 0x7ff7680ccfd0 2024-04-01T00:13:56.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1950179180 con 0x7f19500da9f0 2024-04-01T00:13:56.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe8c0046020 con 0x7fe8c80ceec0 2024-04-01T00:13:56.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1950179740 con 0x7f19500da9f0 2024-04-01T00:13:56.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76815d5f0 con 0x7ff7680ccfd0 2024-04-01T00:13:56.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.650+0000 7fe8beffd640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 0x7fe8ac06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8c00acd30 con 0x7fe8c80ceec0 2024-04-01T00:13:56.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b6040 con 0x7f5df0050190 2024-04-01T00:13:56.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df0180ce0 con 0x7f5df0050190 2024-04-01T00:13:56.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01812a0 con 0x7f5df0050190 2024-04-01T00:13:56.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7fe8cdc1f640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 0x7fe8ac06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80b0a50 con 0x7f5df0050190 2024-04-01T00:13:56.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.651+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b0e00 con 0x7f5df0050190 2024-04-01T00:13:56.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.652+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1950007080 con 0x7f19500da9f0 2024-04-01T00:13:56.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.652+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7680d1a90 con 0x7ff7680ccfd0 2024-04-01T00:13:56.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.652+0000 7fe8cdc1f640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 0x7fe8ac06de80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80678f0 tx=0x7fe8c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.652+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff758012b70 con 0x7ff7680ccfd0 2024-04-01T00:13:56.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0005290 con 0x7f5df0050190 2024-04-01T00:13:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.652+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f936401f020 con 0x7f93680df410 2024-04-01T00:13:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fcf18007600 con 0x7fcf240caf70 2024-04-01T00:13:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7ff75effd640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 0x7ff74c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff75808ab70 con 0x7ff7680ccfd0 2024-04-01T00:13:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7ff76db2e640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 0x7ff74c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7fcf0bfff640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 0x7fcf1006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f5df80c3030 con 0x7f5df0050190 2024-04-01T00:13:56.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.653+0000 7f5dee7fc640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 0x7f5ddc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf18057510 con 0x7fcf240caf70 2024-04-01T00:13:56.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd288012950 con 0x7fd27800b780 2024-04-01T00:13:56.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1944005990 con 0x7f19500da9f0 2024-04-01T00:13:56.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fcf237fe640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 0x7fcf1006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fd27effd640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 0x7fd27006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7f1954ff9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 0x7f193c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:56.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7f5dfd6c7640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 0x7f5ddc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd28808bb70 con 0x7fd27800b780 2024-04-01T00:13:56.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fd28dd9b640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 0x7fd27006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7fcf237fe640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 0x7fcf1006de50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf14036000 tx=0x7fcf1403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7ff76db2e640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 0x7ff74c06df20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7680b4880 tx=0x7ff764007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f194408ae50 con 0x7f19500da9f0 2024-04-01T00:13:56.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.654+0000 7f1957fff640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 0x7f193c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:56.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.655+0000 7fd28dd9b640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 0x7fd27006df20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2900cc290 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.655+0000 7f1957fff640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 0x7f193c06de80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f194c034000 tx=0x7f194c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.655+0000 7f5dfd6c7640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 0x7f5ddc06de50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5df004af70 tx=0x7f5de8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:56.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.656+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fcf18059b70 con 0x7fcf240caf70 2024-04-01T00:13:56.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.659+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe8c004d030 con 0x7fe8c80ceec0 2024-04-01T00:13:56.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.659+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1950007080 con 0x7f19500da9f0 2024-04-01T00:13:56.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.661+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5df80f4670 con 0x7f5df0050190 2024-04-01T00:13:56.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.661+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd28805a760 con 0x7fd27800b780 2024-04-01T00:13:56.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.665+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f5df80ca020 con 0x7f5df0050190 2024-04-01T00:13:56.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:56.669+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff7680d1a90 con 0x7ff7680ccfd0 2024-04-01T00:13:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.062+0000 7f936ffff640 1 -- 172.21.15.64:0/1655664606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9368005380 con 0x7f93680df410 2024-04-01T00:13:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.062+0000 7f9357fff640 1 -- 172.21.15.64:0/1655664606 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9364059280 con 0x7f93680df410 2024-04-01T00:13:57.063 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.066+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 msgr2=0x7f935c06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.066+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 0x7f935c06df20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9360039000 tx=0x7f9360034000 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.066+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 msgr2=0x7f93680d9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.066+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f936400b410 tx=0x7f936400b8d0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.066+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 shutdown_connections 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935c06ba60 0x7f935c06df20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680df410 0x7f93680d9e90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9368054260 0x7f93680485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 --2- 172.21.15.64:0/1655664606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936804fa80 0x7f9368180120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 >> 172.21.15.64:0/1655664606 conn(0x7f9368009fa0 msgr2=0x7f93680372b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 shutdown_connections 2024-04-01T00:13:57.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.067+0000 7f9355ffb640 1 -- 172.21.15.64:0/1655664606 wait complete. 2024-04-01T00:13:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.075+0000 7fde4e9c9640 1 -- 172.21.15.64:0/3044397978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fde28005580 con 0x7fde48071590 2024-04-01T00:13:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.075+0000 7fde44ff9640 1 -- 172.21.15.64:0/3044397978 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fde3c05c0a0 con 0x7fde48071590 2024-04-01T00:13:57.076 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 msgr2=0x7fde3406df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 0x7fde3406df20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde38036000 tx=0x7fde3803c040 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 msgr2=0x7fde4814f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c430 tx=0x7fde3c00c8f0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 shutdown_connections 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde3406ba60 0x7fde3406df20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48162740 0x7fde480c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde480caaf0 0x7fde4814fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.079+0000 7fde2e7fc640 1 --2- 172.21.15.64:0/3044397978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde48071590 0x7fde4814f810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.080+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 >> 172.21.15.64:0/3044397978 conn(0x7fde480cfa90 msgr2=0x7fde48154090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.080+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 shutdown_connections 2024-04-01T00:13:57.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.080+0000 7fde2e7fc640 1 -- 172.21.15.64:0/3044397978 wait complete. 2024-04-01T00:13:57.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.082+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcf240681a0 con 0x7fcf240caf70 2024-04-01T00:13:57.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.083+0000 7fcf0bfff640 1 -- 172.21.15.64:0/1980622097 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf18059470 con 0x7fcf240caf70 2024-04-01T00:13:57.084 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:57.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.084+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe8c80c6410 con 0x7fe8c80ceec0 2024-04-01T00:13:57.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-01T00:13:57.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.085+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd2900681a0 con 0x7fd27800b780 2024-04-01T00:13:57.087 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.085+0000 7fe8beffd640 1 -- 172.21.15.64:0/388452122 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8c007b380 con 0x7fe8c80ceec0 2024-04-01T00:13:57.087 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:57.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.085+0000 7fd27effd640 1 -- 172.21.15.64:0/3783528696 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd28805a060 con 0x7fd27800b780 2024-04-01T00:13:57.088 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 msgr2=0x7fcf1006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 0x7fcf1006de50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf14036000 tx=0x7fcf1403c040 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 msgr2=0x7fcf24144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c490 tx=0x7fcf1800c950 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 shutdown_connections 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf1006b990 0x7fcf1006de50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf24159fc0 0x7fcf24201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf240caf70 0x7fcf24144230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 --2- 172.21.15.64:0/1980622097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240c5880 0x7fcf241fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 >> 172.21.15.64:0/1980622097 conn(0x7fcf240cfa90 msgr2=0x7fcf240cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 shutdown_connections 2024-04-01T00:13:57.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.086+0000 7fcf29e3b640 1 -- 172.21.15.64:0/1980622097 wait complete. 2024-04-01T00:13:57.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.087+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff76804f4b0 con 0x7ff7680ccfd0 2024-04-01T00:13:57.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.087+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f19500d5fc0 con 0x7f19500da9f0 2024-04-01T00:13:57.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.087+0000 7f1954ff9640 1 -- 172.21.15.64:0/3954257314 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1944017030 con 0x7f19500da9f0 2024-04-01T00:13:57.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.087+0000 7ff75effd640 1 -- 172.21.15.64:0/2273300498 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff758059760 con 0x7ff7680ccfd0 2024-04-01T00:13:57.091 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:57.091 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 msgr2=0x7fd27006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 0x7fd27006df20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2900cc290 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 msgr2=0x7fd2901515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 0x7fd2901515c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd28800dc60 tx=0x7fd28800b4a0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 msgr2=0x7fe8ac06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 0x7fe8ac06de80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80678f0 tx=0x7fe8c4009040 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 msgr2=0x7fe8c814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002c4f0 tx=0x7fe8c0004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 shutdown_connections 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe8ac06b9c0 0x7fe8ac06de80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8c80c6b60 0x7fe8c8164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8c81666b0 0x7fe8c80b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 --2- 172.21.15.64:0/388452122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c80ceec0 0x7fe8c814fe40 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002c4f0 tx=0x7fe8c0004a40 comp rx=0 tx=0).stop 2024-04-01T00:13:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 >> 172.21.15.64:0/388452122 conn(0x7fe8c80cfa90 msgr2=0x7fe8c80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 shutdown_connections 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd27006ba60 0x7fd27006df20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29014f6c0 0x7fd2901e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd29014ff80 0x7fd2900c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 --2- 172.21.15.64:0/3783528696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd27800b780 0x7fd2901515c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 >> 172.21.15.64:0/3783528696 conn(0x7fd2900cfa90 msgr2=0x7fd290072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 shutdown_connections 2024-04-01T00:13:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 shutdown_connections 2024-04-01T00:13:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fe8d0684640 1 -- 172.21.15.64:0/388452122 wait complete. 2024-04-01T00:13:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.089+0000 7fd294b77640 1 -- 172.21.15.64:0/3783528696 wait complete. 2024-04-01T00:13:57.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 msgr2=0x7ff74c06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 0x7ff74c06df20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7680b4880 tx=0x7ff764007e00 comp rx=0 tx=0).stop 2024-04-01T00:13:57.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 msgr2=0x7ff7681505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff75800d5f0 tx=0x7ff75800dab0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 msgr2=0x7f193c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 0x7f193c06de80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f194c034000 tx=0x7f194c009030 comp rx=0 tx=0).stop 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 msgr2=0x7f19500dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f194400f930 tx=0x7f194400fd30 comp rx=0 tx=0).stop 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 shutdown_connections 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f193c06b9c0 0x7f193c06de80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195004bb80 0x7f195004c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19500da9f0 0x7f19500dae70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 shutdown_connections 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff74c06ba60 0x7ff74c06df20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76815be70 0x7ff76815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7680d1c70 0x7ff7680c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 --2- 172.21.15.64:0/2273300498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7680ccfd0 0x7ff7681505e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 >> 172.21.15.64:0/2273300498 conn(0x7ff76809a980 msgr2=0x7ff76806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 --2- 172.21.15.64:0/3954257314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1950168d60 0x7f1950169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.091+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 >> 172.21.15.64:0/3954257314 conn(0x7f1950009fa0 msgr2=0x7f19500e16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 shutdown_connections 2024-04-01T00:13:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 shutdown_connections 2024-04-01T00:13:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7ff76f331640 1 -- 172.21.15.64:0/2273300498 wait complete. 2024-04-01T00:13:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.092+0000 7f195d0e9640 1 -- 172.21.15.64:0/3954257314 wait complete. 2024-04-01T00:13:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.093+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5de0005580 con 0x7f5df0050190 2024-04-01T00:13:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.093+0000 7f5dee7fc640 1 -- 172.21.15.64:0/155572176 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5df806f050 con 0x7f5df0050190 2024-04-01T00:13:57.101 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-04-01T00:13:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.096+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 msgr2=0x7f5ddc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.096+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 0x7f5ddc06de50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5df004af70 tx=0x7f5de8009190 comp rx=0 tx=0).stop 2024-04-01T00:13:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.096+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 msgr2=0x7f5df00dd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.096+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5df8050040 tx=0x7f5df8068f20 comp rx=0 tx=0).stop 2024-04-01T00:13:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.096+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 shutdown_connections 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5ddc06b990 0x7f5ddc06de50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df00df210 0x7f5df017d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5df0053c60 0x7f5df0180570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 --2- 172.21.15.64:0/155572176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df0050190 0x7f5df00dd160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 >> 172.21.15.64:0/155572176 conn(0x7f5df0009fa0 msgr2=0x7f5df004bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 shutdown_connections 2024-04-01T00:13:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.097+0000 7f5dfeeca640 1 -- 172.21.15.64:0/155572176 wait complete. 2024-04-01T00:13:57.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-01T00:13:57.104 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-01T00:13:57.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-01T00:13:57.111 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.111 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-04-01T00:13:57.113 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-04-01T00:13:57.113 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-01T00:13:57.115 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-01T00:13:57.116 DEBUG:teuthology.parallel:result is None 2024-04-01T00:13:57.116 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-01T00:13:57.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.255+0000 7f969008f640 1 Processor -- start 2024-04-01T00:13:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.255+0000 7f969008f640 1 -- start start 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f969008f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f96880d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f969008f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96880d0060 con 0x7f96880cc340 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f969008f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968806d100 con 0x7f968806caa0 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f969008f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96880d0650 con 0x7f968815e1c0 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968d62a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f96880cc340 0x7f968806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48052/0 (socket says 172.21.15.64:48052) 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968d62a640 1 -- 172.21.15.64:0/119836400 learned_addr learned my addr 172.21.15.64:0/119836400 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968de2b640 1 --2- 172.21.15.64:0/119836400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f96880d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1244378383 0 0) 0x7f9678001110 con 0x7f96880cc340 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9674003520 con 0x7f96880cc340 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112583480 0 0) 0x7f968806d100 con 0x7f968806caa0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9678001110 con 0x7f968806caa0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968de2b640 1 -- 172.21.15.64:0/119836400 >> v1:172.21.15.100:6789/0 conn(0x7f968806caa0 legacy=0x7f96880b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968de2b640 1 -- 172.21.15.64:0/119836400 >> v1:172.21.15.64:6789/0 conn(0x7f96880cc340 legacy=0x7f968806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.256+0000 7f968de2b640 1 -- 172.21.15.64:0/119836400 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968806d3e0 con 0x7f968815e1c0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.257+0000 7f968de2b640 1 --2- 172.21.15.64:0/119836400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f96880d1c10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f96800097d0 tx=0x7f968002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9734e2fbc572f77d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.257+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680035020 con 0x7f968815e1c0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.257+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9680031800 con 0x7f968815e1c0 2024-04-01T00:13:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.257+0000 7f968ce29640 1 -- 172.21.15.64:0/119836400 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680031bd0 con 0x7f968815e1c0 2024-04-01T00:13:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 -- 172.21.15.64:0/119836400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 msgr2=0x7f96880d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 --2- 172.21.15.64:0/119836400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f96880d1c10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f96800097d0 tx=0x7f968002b8a0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 -- 172.21.15.64:0/119836400 shutdown_connections 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 --2- 172.21.15.64:0/119836400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f96880d1c10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 -- 172.21.15.64:0/119836400 >> 172.21.15.64:0/119836400 conn(0x7f968809a980 msgr2=0x7f968809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 -- 172.21.15.64:0/119836400 shutdown_connections 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.258+0000 7f969008f640 1 -- 172.21.15.64:0/119836400 wait complete. 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.259+0000 7f969008f640 1 Processor -- start 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.259+0000 7f969008f640 1 -- start start 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 0x7f96880c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f9688160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96880d22d0 con 0x7f96880cc340 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96880d2150 con 0x7f968806caa0 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f969008f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96880d2450 con 0x7f968815e1c0 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968de2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 0x7f96880c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968e62c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f9688160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51334/0 (socket says 172.21.15.64:51334) 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968de2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 0x7f96880c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45126/0 (socket says 172.21.15.64:45126) 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 -- 172.21.15.64:0/3982865489 learned_addr learned my addr 172.21.15.64:0/3982865489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 -- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 msgr2=0x7f9688160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f9688160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 -- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 msgr2=0x7f96880c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 0x7f96880c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 -- 172.21.15.64:0/3982865489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96881524a0 con 0x7f96880cc340 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f968d62a640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f967800c730 tx=0x7f967800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678017070 con 0x7f96880cc340 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.260+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678005a90 con 0x7f96880cc340 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.261+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968815fc80 con 0x7f96880cc340 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.261+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678007070 con 0x7f96880cc340 2024-04-01T00:13:57.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.261+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96880c6c40 con 0x7f96880cc340 2024-04-01T00:13:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.262+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96880c6e40 con 0x7f96880cc340 2024-04-01T00:13:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.262+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f967801a070 con 0x7f96880cc340 2024-04-01T00:13:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.262+0000 7f967e7fc640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 0x7f966c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.262+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9678089ee0 con 0x7f96880cc340 2024-04-01T00:13:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.262+0000 7f968de2b640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 0x7f966c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.263+0000 7f968de2b640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 0x7f966c06de50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9680036000 tx=0x7f9680008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.267+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f96880c6e40 con 0x7f96880cc340 2024-04-01T00:13:57.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.443+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96880681a0 con 0x7f966c06b990 2024-04-01T00:13:57.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.449+0000 7f967e7fc640 1 -- 172.21.15.64:0/3982865489 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f96800877d0 con 0x7f966c06b990 2024-04-01T00:13:57.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 msgr2=0x7f966c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 0x7f966c06de50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9680036000 tx=0x7f9680008030 comp rx=0 tx=0).stop 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 msgr2=0x7f9688151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f967800c730 tx=0x7f967800cbf0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 shutdown_connections 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966c06b990 0x7f966c06de50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968815e1c0 0x7f9688160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96880cc340 0x7f9688151f60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 --2- 172.21.15.64:0/3982865489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968806caa0 0x7f96880c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 >> 172.21.15.64:0/3982865489 conn(0x7f968809a980 msgr2=0x7f96880cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 shutdown_connections 2024-04-01T00:13:57.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.452+0000 7f969008f640 1 -- 172.21.15.64:0/3982865489 wait complete. 2024-04-01T00:13:57.464 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-01T00:13:56.747821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.731501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905422+0000","last_change":"2024-04-01T00:13:47.045277+0000","last_active":"2024-04-01T00:13:48.905422+0000","last_peered":"2024-04-01T00:13:48.905422+0000","last_clean":"2024-04-01T00:13:48.905422+0000","last_became_active":"2024-04-01T00:13:47.044728+0000","last_became_peered":"2024-04-01T00:13:47.044728+0000","last_unstale":"2024-04-01T00:13:48.905422+0000","last_undegraded":"2024-04-01T00:13:48.905422+0000","last_fullsized":"2024-04-01T00:13:48.905422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:50:54.919900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025542+0000","last_change":"2024-04-01T00:13:47.039324+0000","last_active":"2024-04-01T00:13:48.025542+0000","last_peered":"2024-04-01T00:13:48.025542+0000","last_clean":"2024-04-01T00:13:48.025542+0000","last_became_active":"2024-04-01T00:13:47.039002+0000","last_became_peered":"2024-04-01T00:13:47.039002+0000","last_unstale":"2024-04-01T00:13:48.025542+0000","last_undegraded":"2024-04-01T00:13:48.025542+0000","last_fullsized":"2024-04-01T00:13:48.025542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906198+0000","last_change":"2024-04-01T00:13:47.039670+0000","last_active":"2024-04-01T00:13:48.906198+0000","last_peered":"2024-04-01T00:13:48.906198+0000","last_clean":"2024-04-01T00:13:48.906198+0000","last_became_active":"2024-04-01T00:13:47.037096+0000","last_became_peered":"2024-04-01T00:13:47.037096+0000","last_unstale":"2024-04-01T00:13:48.906198+0000","last_undegraded":"2024-04-01T00:13:48.906198+0000","last_fullsized":"2024-04-01T00:13:48.906198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:05:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026858+0000","last_change":"2024-04-01T00:13:47.033997+0000","last_active":"2024-04-01T00:13:48.026858+0000","last_peered":"2024-04-01T00:13:48.026858+0000","last_clean":"2024-04-01T00:13:48.026858+0000","last_became_active":"2024-04-01T00:13:47.033682+0000","last_became_peered":"2024-04-01T00:13:47.033682+0000","last_unstale":"2024-04-01T00:13:48.026858+0000","last_undegraded":"2024-04-01T00:13:48.026858+0000","last_fullsized":"2024-04-01T00:13:48.026858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905169+0000","last_change":"2024-04-01T00:13:47.040873+0000","last_active":"2024-04-01T00:13:48.905169+0000","last_peered":"2024-04-01T00:13:48.905169+0000","last_clean":"2024-04-01T00:13:48.905169+0000","last_became_active":"2024-04-01T00:13:47.039953+0000","last_became_peered":"2024-04-01T00:13:47.039953+0000","last_unstale":"2024-04-01T00:13:48.905169+0000","last_undegraded":"2024-04-01T00:13:48.905169+0000","last_fullsized":"2024-04-01T00:13:48.905169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026638+0000","last_change":"2024-04-01T00:13:47.041029+0000","last_active":"2024-04-01T00:13:48.026638+0000","last_peered":"2024-04-01T00:13:48.026638+0000","last_clean":"2024-04-01T00:13:48.026638+0000","last_became_active":"2024-04-01T00:13:47.040771+0000","last_became_peered":"2024-04-01T00:13:47.040771+0000","last_unstale":"2024-04-01T00:13:48.026638+0000","last_undegraded":"2024-04-01T00:13:48.026638+0000","last_fullsized":"2024-04-01T00:13:48.026638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904941+0000","last_change":"2024-04-01T00:13:47.042649+0000","last_active":"2024-04-01T00:13:48.904941+0000","last_peered":"2024-04-01T00:13:48.904941+0000","last_clean":"2024-04-01T00:13:48.904941+0000","last_became_active":"2024-04-01T00:13:47.042373+0000","last_became_peered":"2024-04-01T00:13:47.042373+0000","last_unstale":"2024-04-01T00:13:48.904941+0000","last_undegraded":"2024-04-01T00:13:48.904941+0000","last_fullsized":"2024-04-01T00:13:48.904941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905488+0000","last_change":"2024-04-01T00:13:47.031258+0000","last_active":"2024-04-01T00:13:48.905488+0000","last_peered":"2024-04-01T00:13:48.905488+0000","last_clean":"2024-04-01T00:13:48.905488+0000","last_became_active":"2024-04-01T00:13:47.030972+0000","last_became_peered":"2024-04-01T00:13:47.030972+0000","last_unstale":"2024-04-01T00:13:48.905488+0000","last_undegraded":"2024-04-01T00:13:48.905488+0000","last_fullsized":"2024-04-01T00:13:48.905488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905452+0000","last_change":"2024-04-01T00:13:47.045417+0000","last_active":"2024-04-01T00:13:48.905452+0000","last_peered":"2024-04-01T00:13:48.905452+0000","last_clean":"2024-04-01T00:13:48.905452+0000","last_became_active":"2024-04-01T00:13:47.044978+0000","last_became_peered":"2024-04-01T00:13:47.044978+0000","last_unstale":"2024-04-01T00:13:48.905452+0000","last_undegraded":"2024-04-01T00:13:48.905452+0000","last_fullsized":"2024-04-01T00:13:48.905452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:12:06.404460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030825+0000","last_change":"2024-04-01T00:13:47.035602+0000","last_active":"2024-04-01T00:13:48.030825+0000","last_peered":"2024-04-01T00:13:48.030825+0000","last_clean":"2024-04-01T00:13:48.030825+0000","last_became_active":"2024-04-01T00:13:47.035293+0000","last_became_peered":"2024-04-01T00:13:47.035293+0000","last_unstale":"2024-04-01T00:13:48.030825+0000","last_undegraded":"2024-04-01T00:13:48.030825+0000","last_fullsized":"2024-04-01T00:13:48.030825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030712+0000","last_change":"2024-04-01T00:13:47.037792+0000","last_active":"2024-04-01T00:13:48.030712+0000","last_peered":"2024-04-01T00:13:48.030712+0000","last_clean":"2024-04-01T00:13:48.030712+0000","last_became_active":"2024-04-01T00:13:47.037508+0000","last_became_peered":"2024-04-01T00:13:47.037508+0000","last_unstale":"2024-04-01T00:13:48.030712+0000","last_undegraded":"2024-04-01T00:13:48.030712+0000","last_fullsized":"2024-04-01T00:13:48.030712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028123+0000","last_change":"2024-04-01T00:13:47.026838+0000","last_active":"2024-04-01T00:13:48.028123+0000","last_peered":"2024-04-01T00:13:48.028123+0000","last_clean":"2024-04-01T00:13:48.028123+0000","last_became_active":"2024-04-01T00:13:47.026586+0000","last_became_peered":"2024-04-01T00:13:47.026586+0000","last_unstale":"2024-04-01T00:13:48.028123+0000","last_undegraded":"2024-04-01T00:13:48.028123+0000","last_fullsized":"2024-04-01T00:13:48.028123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:13.513798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905964+0000","last_change":"2024-04-01T00:13:47.028848+0000","last_active":"2024-04-01T00:13:48.905964+0000","last_peered":"2024-04-01T00:13:48.905964+0000","last_clean":"2024-04-01T00:13:48.905964+0000","last_became_active":"2024-04-01T00:13:47.028577+0000","last_became_peered":"2024-04-01T00:13:47.028577+0000","last_unstale":"2024-04-01T00:13:48.905964+0000","last_undegraded":"2024-04-01T00:13:48.905964+0000","last_fullsized":"2024-04-01T00:13:48.905964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906018+0000","last_change":"2024-04-01T00:13:47.039847+0000","last_active":"2024-04-01T00:13:48.906018+0000","last_peered":"2024-04-01T00:13:48.906018+0000","last_clean":"2024-04-01T00:13:48.906018+0000","last_became_active":"2024-04-01T00:13:47.039459+0000","last_became_peered":"2024-04-01T00:13:47.039459+0000","last_unstale":"2024-04-01T00:13:48.906018+0000","last_undegraded":"2024-04-01T00:13:48.906018+0000","last_fullsized":"2024-04-01T00:13:48.906018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:51.654019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025413+0000","last_change":"2024-04-01T00:13:47.034288+0000","last_active":"2024-04-01T00:13:48.025413+0000","last_peered":"2024-04-01T00:13:48.025413+0000","last_clean":"2024-04-01T00:13:48.025413+0000","last_became_active":"2024-04-01T00:13:47.034022+0000","last_became_peered":"2024-04-01T00:13:47.034022+0000","last_unstale":"2024-04-01T00:13:48.025413+0000","last_undegraded":"2024-04-01T00:13:48.025413+0000","last_fullsized":"2024-04-01T00:13:48.025413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026557+0000","last_change":"2024-04-01T00:13:47.039173+0000","last_active":"2024-04-01T00:13:48.026557+0000","last_peered":"2024-04-01T00:13:48.026557+0000","last_clean":"2024-04-01T00:13:48.026557+0000","last_became_active":"2024-04-01T00:13:47.038845+0000","last_became_peered":"2024-04-01T00:13:47.038845+0000","last_unstale":"2024-04-01T00:13:48.026557+0000","last_undegraded":"2024-04-01T00:13:48.026557+0000","last_fullsized":"2024-04-01T00:13:48.026557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027411+0000","last_change":"2024-04-01T00:13:47.043400+0000","last_active":"2024-04-01T00:13:48.027411+0000","last_peered":"2024-04-01T00:13:48.027411+0000","last_clean":"2024-04-01T00:13:48.027411+0000","last_became_active":"2024-04-01T00:13:47.043120+0000","last_became_peered":"2024-04-01T00:13:47.043120+0000","last_unstale":"2024-04-01T00:13:48.027411+0000","last_undegraded":"2024-04-01T00:13:48.027411+0000","last_fullsized":"2024-04-01T00:13:48.027411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905429+0000","last_change":"2024-04-01T00:13:47.035436+0000","last_active":"2024-04-01T00:13:48.905429+0000","last_peered":"2024-04-01T00:13:48.905429+0000","last_clean":"2024-04-01T00:13:48.905429+0000","last_became_active":"2024-04-01T00:13:47.035072+0000","last_became_peered":"2024-04-01T00:13:47.035072+0000","last_unstale":"2024-04-01T00:13:48.905429+0000","last_undegraded":"2024-04-01T00:13:48.905429+0000","last_fullsized":"2024-04-01T00:13:48.905429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905379+0000","last_change":"2024-04-01T00:13:47.040506+0000","last_active":"2024-04-01T00:13:48.905379+0000","last_peered":"2024-04-01T00:13:48.905379+0000","last_clean":"2024-04-01T00:13:48.905379+0000","last_became_active":"2024-04-01T00:13:47.040244+0000","last_became_peered":"2024-04-01T00:13:47.040244+0000","last_unstale":"2024-04-01T00:13:48.905379+0000","last_undegraded":"2024-04-01T00:13:48.905379+0000","last_fullsized":"2024-04-01T00:13:48.905379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030707+0000","last_change":"2024-04-01T00:13:47.031532+0000","last_active":"2024-04-01T00:13:48.030707+0000","last_peered":"2024-04-01T00:13:48.030707+0000","last_clean":"2024-04-01T00:13:48.030707+0000","last_became_active":"2024-04-01T00:13:47.031252+0000","last_became_peered":"2024-04-01T00:13:47.031252+0000","last_unstale":"2024-04-01T00:13:48.030707+0000","last_undegraded":"2024-04-01T00:13:48.030707+0000","last_fullsized":"2024-04-01T00:13:48.030707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906466+0000","last_change":"2024-04-01T00:13:47.047404+0000","last_active":"2024-04-01T00:13:48.906466+0000","last_peered":"2024-04-01T00:13:48.906466+0000","last_clean":"2024-04-01T00:13:48.906466+0000","last_became_active":"2024-04-01T00:13:47.047080+0000","last_became_peered":"2024-04-01T00:13:47.047080+0000","last_unstale":"2024-04-01T00:13:48.906466+0000","last_undegraded":"2024-04-01T00:13:48.906466+0000","last_fullsized":"2024-04-01T00:13:48.906466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026698+0000","last_change":"2024-04-01T00:13:47.033044+0000","last_active":"2024-04-01T00:13:48.026698+0000","last_peered":"2024-04-01T00:13:48.026698+0000","last_clean":"2024-04-01T00:13:48.026698+0000","last_became_active":"2024-04-01T00:13:47.032783+0000","last_became_peered":"2024-04-01T00:13:47.032783+0000","last_unstale":"2024-04-01T00:13:48.026698+0000","last_undegraded":"2024-04-01T00:13:48.026698+0000","last_fullsized":"2024-04-01T00:13:48.026698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905485+0000","last_change":"2024-04-01T00:13:47.035282+0000","last_active":"2024-04-01T00:13:48.905485+0000","last_peered":"2024-04-01T00:13:48.905485+0000","last_clean":"2024-04-01T00:13:48.905485+0000","last_became_active":"2024-04-01T00:13:47.034953+0000","last_became_peered":"2024-04-01T00:13:47.034953+0000","last_unstale":"2024-04-01T00:13:48.905485+0000","last_undegraded":"2024-04-01T00:13:48.905485+0000","last_fullsized":"2024-04-01T00:13:48.905485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026661+0000","last_change":"2024-04-01T00:13:47.034177+0000","last_active":"2024-04-01T00:13:48.026661+0000","last_peered":"2024-04-01T00:13:48.026661+0000","last_clean":"2024-04-01T00:13:48.026661+0000","last_became_active":"2024-04-01T00:13:47.033426+0000","last_became_peered":"2024-04-01T00:13:47.033426+0000","last_unstale":"2024-04-01T00:13:48.026661+0000","last_undegraded":"2024-04-01T00:13:48.026661+0000","last_fullsized":"2024-04-01T00:13:48.026661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906196+0000","last_change":"2024-04-01T00:13:47.034202+0000","last_active":"2024-04-01T00:13:48.906196+0000","last_peered":"2024-04-01T00:13:48.906196+0000","last_clean":"2024-04-01T00:13:48.906196+0000","last_became_active":"2024-04-01T00:13:47.033941+0000","last_became_peered":"2024-04-01T00:13:47.033941+0000","last_unstale":"2024-04-01T00:13:48.906196+0000","last_undegraded":"2024-04-01T00:13:48.906196+0000","last_fullsized":"2024-04-01T00:13:48.906196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030886+0000","last_change":"2024-04-01T00:13:47.042844+0000","last_active":"2024-04-01T00:13:48.030886+0000","last_peered":"2024-04-01T00:13:48.030886+0000","last_clean":"2024-04-01T00:13:48.030886+0000","last_became_active":"2024-04-01T00:13:47.042518+0000","last_became_peered":"2024-04-01T00:13:47.042518+0000","last_unstale":"2024-04-01T00:13:48.030886+0000","last_undegraded":"2024-04-01T00:13:48.030886+0000","last_fullsized":"2024-04-01T00:13:48.030886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025478+0000","last_change":"2024-04-01T00:13:47.032968+0000","last_active":"2024-04-01T00:13:48.025478+0000","last_peered":"2024-04-01T00:13:48.025478+0000","last_clean":"2024-04-01T00:13:48.025478+0000","last_became_active":"2024-04-01T00:13:47.032695+0000","last_became_peered":"2024-04-01T00:13:47.032695+0000","last_unstale":"2024-04-01T00:13:48.025478+0000","last_undegraded":"2024-04-01T00:13:48.025478+0000","last_fullsized":"2024-04-01T00:13:48.025478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906706+0000","last_change":"2024-04-01T00:13:47.035447+0000","last_active":"2024-04-01T00:13:48.906706+0000","last_peered":"2024-04-01T00:13:48.906706+0000","last_clean":"2024-04-01T00:13:48.906706+0000","last_became_active":"2024-04-01T00:13:47.035003+0000","last_became_peered":"2024-04-01T00:13:47.035003+0000","last_unstale":"2024-04-01T00:13:48.906706+0000","last_undegraded":"2024-04-01T00:13:48.906706+0000","last_fullsized":"2024-04-01T00:13:48.906706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906541+0000","last_change":"2024-04-01T00:13:47.039218+0000","last_active":"2024-04-01T00:13:48.906541+0000","last_peered":"2024-04-01T00:13:48.906541+0000","last_clean":"2024-04-01T00:13:48.906541+0000","last_became_active":"2024-04-01T00:13:47.037984+0000","last_became_peered":"2024-04-01T00:13:47.037984+0000","last_unstale":"2024-04-01T00:13:48.906541+0000","last_undegraded":"2024-04-01T00:13:48.906541+0000","last_fullsized":"2024-04-01T00:13:48.906541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026792+0000","last_change":"2024-04-01T00:13:47.034521+0000","last_active":"2024-04-01T00:13:48.026792+0000","last_peered":"2024-04-01T00:13:48.026792+0000","last_clean":"2024-04-01T00:13:48.026792+0000","last_became_active":"2024-04-01T00:13:47.034251+0000","last_became_peered":"2024-04-01T00:13:47.034251+0000","last_unstale":"2024-04-01T00:13:48.026792+0000","last_undegraded":"2024-04-01T00:13:48.026792+0000","last_fullsized":"2024-04-01T00:13:48.026792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905386+0000","last_change":"2024-04-01T00:13:47.035817+0000","last_active":"2024-04-01T00:13:48.905386+0000","last_peered":"2024-04-01T00:13:48.905386+0000","last_clean":"2024-04-01T00:13:48.905386+0000","last_became_active":"2024-04-01T00:13:47.034834+0000","last_became_peered":"2024-04-01T00:13:47.034834+0000","last_unstale":"2024-04-01T00:13:48.905386+0000","last_undegraded":"2024-04-01T00:13:48.905386+0000","last_fullsized":"2024-04-01T00:13:48.905386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030497+0000","last_change":"2024-04-01T00:13:47.039829+0000","last_active":"2024-04-01T00:13:48.030497+0000","last_peered":"2024-04-01T00:13:48.030497+0000","last_clean":"2024-04-01T00:13:48.030497+0000","last_became_active":"2024-04-01T00:13:47.039517+0000","last_became_peered":"2024-04-01T00:13:47.039517+0000","last_unstale":"2024-04-01T00:13:48.030497+0000","last_undegraded":"2024-04-01T00:13:48.030497+0000","last_fullsized":"2024-04-01T00:13:48.030497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027883+0000","last_change":"2024-04-01T00:13:47.026124+0000","last_active":"2024-04-01T00:13:48.027883+0000","last_peered":"2024-04-01T00:13:48.027883+0000","last_clean":"2024-04-01T00:13:48.027883+0000","last_became_active":"2024-04-01T00:13:47.025855+0000","last_became_peered":"2024-04-01T00:13:47.025855+0000","last_unstale":"2024-04-01T00:13:48.027883+0000","last_undegraded":"2024-04-01T00:13:48.027883+0000","last_fullsized":"2024-04-01T00:13:48.027883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:56.178747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906229+0000","last_change":"2024-04-01T00:13:46.037694+0000","last_active":"2024-04-01T00:13:48.906229+0000","last_peered":"2024-04-01T00:13:48.906229+0000","last_clean":"2024-04-01T00:13:48.906229+0000","last_became_active":"2024-04-01T00:13:46.037390+0000","last_became_peered":"2024-04-01T00:13:46.037390+0000","last_unstale":"2024-04-01T00:13:48.906229+0000","last_undegraded":"2024-04-01T00:13:48.906229+0000","last_fullsized":"2024-04-01T00:13:48.906229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906550+0000","last_change":"2024-04-01T00:13:47.044544+0000","last_active":"2024-04-01T00:13:48.906550+0000","last_peered":"2024-04-01T00:13:48.906550+0000","last_clean":"2024-04-01T00:13:48.906550+0000","last_became_active":"2024-04-01T00:13:47.044266+0000","last_became_peered":"2024-04-01T00:13:47.044266+0000","last_unstale":"2024-04-01T00:13:48.906550+0000","last_undegraded":"2024-04-01T00:13:48.906550+0000","last_fullsized":"2024-04-01T00:13:48.906550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030621+0000","last_change":"2024-04-01T00:13:46.036332+0000","last_active":"2024-04-01T00:13:48.030621+0000","last_peered":"2024-04-01T00:13:48.030621+0000","last_clean":"2024-04-01T00:13:48.030621+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.030621+0000","last_undegraded":"2024-04-01T00:13:48.030621+0000","last_fullsized":"2024-04-01T00:13:48.030621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905834+0000","last_change":"2024-04-01T00:13:46.032078+0000","last_active":"2024-04-01T00:13:48.905834+0000","last_peered":"2024-04-01T00:13:48.905834+0000","last_clean":"2024-04-01T00:13:48.905834+0000","last_became_active":"2024-04-01T00:13:46.031705+0000","last_became_peered":"2024-04-01T00:13:46.031705+0000","last_unstale":"2024-04-01T00:13:48.905834+0000","last_undegraded":"2024-04-01T00:13:48.905834+0000","last_fullsized":"2024-04-01T00:13:48.905834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905625+0000","last_change":"2024-04-01T00:13:47.041444+0000","last_active":"2024-04-01T00:13:48.905625+0000","last_peered":"2024-04-01T00:13:48.905625+0000","last_clean":"2024-04-01T00:13:48.905625+0000","last_became_active":"2024-04-01T00:13:47.041163+0000","last_became_peered":"2024-04-01T00:13:47.041163+0000","last_unstale":"2024-04-01T00:13:48.905625+0000","last_undegraded":"2024-04-01T00:13:48.905625+0000","last_fullsized":"2024-04-01T00:13:48.905625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026649+0000","last_change":"2024-04-01T00:13:46.030376+0000","last_active":"2024-04-01T00:13:48.026649+0000","last_peered":"2024-04-01T00:13:48.026649+0000","last_clean":"2024-04-01T00:13:48.026649+0000","last_became_active":"2024-04-01T00:13:46.030085+0000","last_became_peered":"2024-04-01T00:13:46.030085+0000","last_unstale":"2024-04-01T00:13:48.026649+0000","last_undegraded":"2024-04-01T00:13:48.026649+0000","last_fullsized":"2024-04-01T00:13:48.026649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025414+0000","last_change":"2024-04-01T00:13:47.032819+0000","last_active":"2024-04-01T00:13:48.025414+0000","last_peered":"2024-04-01T00:13:48.025414+0000","last_clean":"2024-04-01T00:13:48.025414+0000","last_became_active":"2024-04-01T00:13:47.032501+0000","last_became_peered":"2024-04-01T00:13:47.032501+0000","last_unstale":"2024-04-01T00:13:48.025414+0000","last_undegraded":"2024-04-01T00:13:48.025414+0000","last_fullsized":"2024-04-01T00:13:48.025414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030703+0000","last_change":"2024-04-01T00:13:46.030622+0000","last_active":"2024-04-01T00:13:48.030703+0000","last_peered":"2024-04-01T00:13:48.030703+0000","last_clean":"2024-04-01T00:13:48.030703+0000","last_became_active":"2024-04-01T00:13:46.030340+0000","last_became_peered":"2024-04-01T00:13:46.030340+0000","last_unstale":"2024-04-01T00:13:48.030703+0000","last_undegraded":"2024-04-01T00:13:48.030703+0000","last_fullsized":"2024-04-01T00:13:48.030703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027187+0000","last_change":"2024-04-01T00:13:47.042195+0000","last_active":"2024-04-01T00:13:48.027187+0000","last_peered":"2024-04-01T00:13:48.027187+0000","last_clean":"2024-04-01T00:13:48.027187+0000","last_became_active":"2024-04-01T00:13:47.041925+0000","last_became_peered":"2024-04-01T00:13:47.041925+0000","last_unstale":"2024-04-01T00:13:48.027187+0000","last_undegraded":"2024-04-01T00:13:48.027187+0000","last_fullsized":"2024-04-01T00:13:48.027187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027178+0000","last_change":"2024-04-01T00:13:46.029157+0000","last_active":"2024-04-01T00:13:48.027178+0000","last_peered":"2024-04-01T00:13:48.027178+0000","last_clean":"2024-04-01T00:13:48.027178+0000","last_became_active":"2024-04-01T00:13:46.028917+0000","last_became_peered":"2024-04-01T00:13:46.028917+0000","last_unstale":"2024-04-01T00:13:48.027178+0000","last_undegraded":"2024-04-01T00:13:48.027178+0000","last_fullsized":"2024-04-01T00:13:48.027178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906652+0000","last_change":"2024-04-01T00:13:47.040888+0000","last_active":"2024-04-01T00:13:48.906652+0000","last_peered":"2024-04-01T00:13:48.906652+0000","last_clean":"2024-04-01T00:13:48.906652+0000","last_became_active":"2024-04-01T00:13:47.040621+0000","last_became_peered":"2024-04-01T00:13:47.040621+0000","last_unstale":"2024-04-01T00:13:48.906652+0000","last_undegraded":"2024-04-01T00:13:48.906652+0000","last_fullsized":"2024-04-01T00:13:48.906652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025420+0000","last_change":"2024-04-01T00:13:46.030545+0000","last_active":"2024-04-01T00:13:48.025420+0000","last_peered":"2024-04-01T00:13:48.025420+0000","last_clean":"2024-04-01T00:13:48.025420+0000","last_became_active":"2024-04-01T00:13:46.030232+0000","last_became_peered":"2024-04-01T00:13:46.030232+0000","last_unstale":"2024-04-01T00:13:48.025420+0000","last_undegraded":"2024-04-01T00:13:48.025420+0000","last_fullsized":"2024-04-01T00:13:48.025420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028260+0000","last_change":"2024-04-01T00:13:47.042059+0000","last_active":"2024-04-01T00:13:48.028260+0000","last_peered":"2024-04-01T00:13:48.028260+0000","last_clean":"2024-04-01T00:13:48.028260+0000","last_became_active":"2024-04-01T00:13:47.040047+0000","last_became_peered":"2024-04-01T00:13:47.040047+0000","last_unstale":"2024-04-01T00:13:48.028260+0000","last_undegraded":"2024-04-01T00:13:48.028260+0000","last_fullsized":"2024-04-01T00:13:48.028260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030779+0000","last_change":"2024-04-01T00:13:46.039366+0000","last_active":"2024-04-01T00:13:48.030779+0000","last_peered":"2024-04-01T00:13:48.030779+0000","last_clean":"2024-04-01T00:13:48.030779+0000","last_became_active":"2024-04-01T00:13:46.039132+0000","last_became_peered":"2024-04-01T00:13:46.039132+0000","last_unstale":"2024-04-01T00:13:48.030779+0000","last_undegraded":"2024-04-01T00:13:48.030779+0000","last_fullsized":"2024-04-01T00:13:48.030779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905125+0000","last_change":"2024-04-01T00:13:47.039118+0000","last_active":"2024-04-01T00:13:48.905125+0000","last_peered":"2024-04-01T00:13:48.905125+0000","last_clean":"2024-04-01T00:13:48.905125+0000","last_became_active":"2024-04-01T00:13:47.035193+0000","last_became_peered":"2024-04-01T00:13:47.035193+0000","last_unstale":"2024-04-01T00:13:48.905125+0000","last_undegraded":"2024-04-01T00:13:48.905125+0000","last_fullsized":"2024-04-01T00:13:48.905125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906606+0000","last_change":"2024-04-01T00:13:46.033537+0000","last_active":"2024-04-01T00:13:48.906606+0000","last_peered":"2024-04-01T00:13:48.906606+0000","last_clean":"2024-04-01T00:13:48.906606+0000","last_became_active":"2024-04-01T00:13:46.033174+0000","last_became_peered":"2024-04-01T00:13:46.033174+0000","last_unstale":"2024-04-01T00:13:48.906606+0000","last_undegraded":"2024-04-01T00:13:48.906606+0000","last_fullsized":"2024-04-01T00:13:48.906606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:15:18.515585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906163+0000","last_change":"2024-04-01T00:13:47.046668+0000","last_active":"2024-04-01T00:13:48.906163+0000","last_peered":"2024-04-01T00:13:48.906163+0000","last_clean":"2024-04-01T00:13:48.906163+0000","last_became_active":"2024-04-01T00:13:47.046387+0000","last_became_peered":"2024-04-01T00:13:47.046387+0000","last_unstale":"2024-04-01T00:13:48.906163+0000","last_undegraded":"2024-04-01T00:13:48.906163+0000","last_fullsized":"2024-04-01T00:13:48.906163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:44:37.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026803+0000","last_change":"2024-04-01T00:13:46.026873+0000","last_active":"2024-04-01T00:13:48.026803+0000","last_peered":"2024-04-01T00:13:48.026803+0000","last_clean":"2024-04-01T00:13:48.026803+0000","last_became_active":"2024-04-01T00:13:46.026564+0000","last_became_peered":"2024-04-01T00:13:46.026564+0000","last_unstale":"2024-04-01T00:13:48.026803+0000","last_undegraded":"2024-04-01T00:13:48.026803+0000","last_fullsized":"2024-04-01T00:13:48.026803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905168+0000","last_change":"2024-04-01T00:13:47.040797+0000","last_active":"2024-04-01T00:13:48.905168+0000","last_peered":"2024-04-01T00:13:48.905168+0000","last_clean":"2024-04-01T00:13:48.905168+0000","last_became_active":"2024-04-01T00:13:47.040224+0000","last_became_peered":"2024-04-01T00:13:47.040224+0000","last_unstale":"2024-04-01T00:13:48.905168+0000","last_undegraded":"2024-04-01T00:13:48.905168+0000","last_fullsized":"2024-04-01T00:13:48.905168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030476+0000","last_change":"2024-04-01T00:13:46.036254+0000","last_active":"2024-04-01T00:13:48.030476+0000","last_peered":"2024-04-01T00:13:48.030476+0000","last_clean":"2024-04-01T00:13:48.030476+0000","last_became_active":"2024-04-01T00:13:46.035969+0000","last_became_peered":"2024-04-01T00:13:46.035969+0000","last_unstale":"2024-04-01T00:13:48.030476+0000","last_undegraded":"2024-04-01T00:13:48.030476+0000","last_fullsized":"2024-04-01T00:13:48.030476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030469+0000","last_change":"2024-04-01T00:13:47.039818+0000","last_active":"2024-04-01T00:13:48.030469+0000","last_peered":"2024-04-01T00:13:48.030469+0000","last_clean":"2024-04-01T00:13:48.030469+0000","last_became_active":"2024-04-01T00:13:47.039157+0000","last_became_peered":"2024-04-01T00:13:47.039157+0000","last_unstale":"2024-04-01T00:13:48.030469+0000","last_undegraded":"2024-04-01T00:13:48.030469+0000","last_fullsized":"2024-04-01T00:13:48.030469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905588+0000","last_change":"2024-04-01T00:13:46.023660+0000","last_active":"2024-04-01T00:13:48.905588+0000","last_peered":"2024-04-01T00:13:48.905588+0000","last_clean":"2024-04-01T00:13:48.905588+0000","last_became_active":"2024-04-01T00:13:46.023368+0000","last_became_peered":"2024-04-01T00:13:46.023368+0000","last_unstale":"2024-04-01T00:13:48.905588+0000","last_undegraded":"2024-04-01T00:13:48.905588+0000","last_fullsized":"2024-04-01T00:13:48.905588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:11:34.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904995+0000","last_change":"2024-04-01T00:13:47.040468+0000","last_active":"2024-04-01T00:13:48.904995+0000","last_peered":"2024-04-01T00:13:48.904995+0000","last_clean":"2024-04-01T00:13:48.904995+0000","last_became_active":"2024-04-01T00:13:47.040198+0000","last_became_peered":"2024-04-01T00:13:47.040198+0000","last_unstale":"2024-04-01T00:13:48.904995+0000","last_undegraded":"2024-04-01T00:13:48.904995+0000","last_fullsized":"2024-04-01T00:13:48.904995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028251+0000","last_change":"2024-04-01T00:13:46.029828+0000","last_active":"2024-04-01T00:13:48.028251+0000","last_peered":"2024-04-01T00:13:48.028251+0000","last_clean":"2024-04-01T00:13:48.028251+0000","last_became_active":"2024-04-01T00:13:46.029514+0000","last_became_peered":"2024-04-01T00:13:46.029514+0000","last_unstale":"2024-04-01T00:13:48.028251+0000","last_undegraded":"2024-04-01T00:13:48.028251+0000","last_fullsized":"2024-04-01T00:13:48.028251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:16:56.432398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905327+0000","last_change":"2024-04-01T00:13:47.037521+0000","last_active":"2024-04-01T00:13:48.905327+0000","last_peered":"2024-04-01T00:13:48.905327+0000","last_clean":"2024-04-01T00:13:48.905327+0000","last_became_active":"2024-04-01T00:13:47.037208+0000","last_became_peered":"2024-04-01T00:13:47.037208+0000","last_unstale":"2024-04-01T00:13:48.905327+0000","last_undegraded":"2024-04-01T00:13:48.905327+0000","last_fullsized":"2024-04-01T00:13:48.905327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905466+0000","last_change":"2024-04-01T00:13:46.029659+0000","last_active":"2024-04-01T00:13:48.905466+0000","last_peered":"2024-04-01T00:13:48.905466+0000","last_clean":"2024-04-01T00:13:48.905466+0000","last_became_active":"2024-04-01T00:13:46.029376+0000","last_became_peered":"2024-04-01T00:13:46.029376+0000","last_unstale":"2024-04-01T00:13:48.905466+0000","last_undegraded":"2024-04-01T00:13:48.905466+0000","last_fullsized":"2024-04-01T00:13:48.905466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905503+0000","last_change":"2024-04-01T00:13:47.045515+0000","last_active":"2024-04-01T00:13:48.905503+0000","last_peered":"2024-04-01T00:13:48.905503+0000","last_clean":"2024-04-01T00:13:48.905503+0000","last_became_active":"2024-04-01T00:13:47.045138+0000","last_became_peered":"2024-04-01T00:13:47.045138+0000","last_unstale":"2024-04-01T00:13:48.905503+0000","last_undegraded":"2024-04-01T00:13:48.905503+0000","last_fullsized":"2024-04-01T00:13:48.905503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:32:58.179314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025530+0000","last_change":"2024-04-01T00:13:46.033447+0000","last_active":"2024-04-01T00:13:48.025530+0000","last_peered":"2024-04-01T00:13:48.025530+0000","last_clean":"2024-04-01T00:13:48.025530+0000","last_became_active":"2024-04-01T00:13:46.033126+0000","last_became_peered":"2024-04-01T00:13:46.033126+0000","last_unstale":"2024-04-01T00:13:48.025530+0000","last_undegraded":"2024-04-01T00:13:48.025530+0000","last_fullsized":"2024-04-01T00:13:48.025530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027072+0000","last_change":"2024-04-01T00:13:47.033643+0000","last_active":"2024-04-01T00:13:48.027072+0000","last_peered":"2024-04-01T00:13:48.027072+0000","last_clean":"2024-04-01T00:13:48.027072+0000","last_became_active":"2024-04-01T00:13:47.033350+0000","last_became_peered":"2024-04-01T00:13:47.033350+0000","last_unstale":"2024-04-01T00:13:48.027072+0000","last_undegraded":"2024-04-01T00:13:48.027072+0000","last_fullsized":"2024-04-01T00:13:48.027072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905480+0000","last_change":"2024-04-01T00:13:46.021048+0000","last_active":"2024-04-01T00:13:48.905480+0000","last_peered":"2024-04-01T00:13:48.905480+0000","last_clean":"2024-04-01T00:13:48.905480+0000","last_became_active":"2024-04-01T00:13:46.020612+0000","last_became_peered":"2024-04-01T00:13:46.020612+0000","last_unstale":"2024-04-01T00:13:48.905480+0000","last_undegraded":"2024-04-01T00:13:48.905480+0000","last_fullsized":"2024-04-01T00:13:48.905480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905112+0000","last_change":"2024-04-01T00:13:47.038059+0000","last_active":"2024-04-01T00:13:48.905112+0000","last_peered":"2024-04-01T00:13:48.905112+0000","last_clean":"2024-04-01T00:13:48.905112+0000","last_became_active":"2024-04-01T00:13:47.034213+0000","last_became_peered":"2024-04-01T00:13:47.034213+0000","last_unstale":"2024-04-01T00:13:48.905112+0000","last_undegraded":"2024-04-01T00:13:48.905112+0000","last_fullsized":"2024-04-01T00:13:48.905112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904882+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:48.904882+0000","last_peered":"2024-04-01T00:13:48.904882+0000","last_clean":"2024-04-01T00:13:48.904882+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:48.904882+0000","last_undegraded":"2024-04-01T00:13:48.904882+0000","last_fullsized":"2024-04-01T00:13:48.904882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026668+0000","last_change":"2024-04-01T00:13:46.019635+0000","last_active":"2024-04-01T00:13:48.026668+0000","last_peered":"2024-04-01T00:13:48.026668+0000","last_clean":"2024-04-01T00:13:48.026668+0000","last_became_active":"2024-04-01T00:13:46.019355+0000","last_became_peered":"2024-04-01T00:13:46.019355+0000","last_unstale":"2024-04-01T00:13:48.026668+0000","last_undegraded":"2024-04-01T00:13:48.026668+0000","last_fullsized":"2024-04-01T00:13:48.026668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906283+0000","last_change":"2024-04-01T00:13:47.034055+0000","last_active":"2024-04-01T00:13:48.906283+0000","last_peered":"2024-04-01T00:13:48.906283+0000","last_clean":"2024-04-01T00:13:48.906283+0000","last_became_active":"2024-04-01T00:13:47.033778+0000","last_became_peered":"2024-04-01T00:13:47.033778+0000","last_unstale":"2024-04-01T00:13:48.906283+0000","last_undegraded":"2024-04-01T00:13:48.906283+0000","last_fullsized":"2024-04-01T00:13:48.906283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904818+0000","last_change":"2024-04-01T00:13:46.037663+0000","last_active":"2024-04-01T00:13:48.904818+0000","last_peered":"2024-04-01T00:13:48.904818+0000","last_clean":"2024-04-01T00:13:48.904818+0000","last_became_active":"2024-04-01T00:13:46.037388+0000","last_became_peered":"2024-04-01T00:13:46.037388+0000","last_unstale":"2024-04-01T00:13:48.904818+0000","last_undegraded":"2024-04-01T00:13:48.904818+0000","last_fullsized":"2024-04-01T00:13:48.904818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906330+0000","last_change":"2024-04-01T00:13:47.038068+0000","last_active":"2024-04-01T00:13:48.906330+0000","last_peered":"2024-04-01T00:13:48.906330+0000","last_clean":"2024-04-01T00:13:48.906330+0000","last_became_active":"2024-04-01T00:13:47.037774+0000","last_became_peered":"2024-04-01T00:13:47.037774+0000","last_unstale":"2024-04-01T00:13:48.906330+0000","last_undegraded":"2024-04-01T00:13:48.906330+0000","last_fullsized":"2024-04-01T00:13:48.906330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:17:27.372297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905353+0000","last_change":"2024-04-01T00:13:46.029621+0000","last_active":"2024-04-01T00:13:48.905353+0000","last_peered":"2024-04-01T00:13:48.905353+0000","last_clean":"2024-04-01T00:13:48.905353+0000","last_became_active":"2024-04-01T00:13:46.029319+0000","last_became_peered":"2024-04-01T00:13:46.029319+0000","last_unstale":"2024-04-01T00:13:48.905353+0000","last_undegraded":"2024-04-01T00:13:48.905353+0000","last_fullsized":"2024-04-01T00:13:48.905353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025440+0000","last_change":"2024-04-01T00:13:47.030092+0000","last_active":"2024-04-01T00:13:48.025440+0000","last_peered":"2024-04-01T00:13:48.025440+0000","last_clean":"2024-04-01T00:13:48.025440+0000","last_became_active":"2024-04-01T00:13:47.029837+0000","last_became_peered":"2024-04-01T00:13:47.029837+0000","last_unstale":"2024-04-01T00:13:48.025440+0000","last_undegraded":"2024-04-01T00:13:48.025440+0000","last_fullsized":"2024-04-01T00:13:48.025440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026885+0000","last_change":"2024-04-01T00:13:46.027134+0000","last_active":"2024-04-01T00:13:48.026885+0000","last_peered":"2024-04-01T00:13:48.026885+0000","last_clean":"2024-04-01T00:13:48.026885+0000","last_became_active":"2024-04-01T00:13:46.026855+0000","last_became_peered":"2024-04-01T00:13:46.026855+0000","last_unstale":"2024-04-01T00:13:48.026885+0000","last_undegraded":"2024-04-01T00:13:48.026885+0000","last_fullsized":"2024-04-01T00:13:48.026885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028071+0000","last_change":"2024-04-01T00:13:47.041868+0000","last_active":"2024-04-01T00:13:48.028071+0000","last_peered":"2024-04-01T00:13:48.028071+0000","last_clean":"2024-04-01T00:13:48.028071+0000","last_became_active":"2024-04-01T00:13:47.040849+0000","last_became_peered":"2024-04-01T00:13:47.040849+0000","last_unstale":"2024-04-01T00:13:48.028071+0000","last_undegraded":"2024-04-01T00:13:48.028071+0000","last_fullsized":"2024-04-01T00:13:48.028071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:54.759711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025337+0000","last_change":"2024-04-01T00:13:46.026672+0000","last_active":"2024-04-01T00:13:48.025337+0000","last_peered":"2024-04-01T00:13:48.025337+0000","last_clean":"2024-04-01T00:13:48.025337+0000","last_became_active":"2024-04-01T00:13:46.026365+0000","last_became_peered":"2024-04-01T00:13:46.026365+0000","last_unstale":"2024-04-01T00:13:48.025337+0000","last_undegraded":"2024-04-01T00:13:48.025337+0000","last_fullsized":"2024-04-01T00:13:48.025337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027240+0000","last_change":"2024-04-01T00:13:47.028664+0000","last_active":"2024-04-01T00:13:48.027240+0000","last_peered":"2024-04-01T00:13:48.027240+0000","last_clean":"2024-04-01T00:13:48.027240+0000","last_became_active":"2024-04-01T00:13:47.028387+0000","last_became_peered":"2024-04-01T00:13:47.028387+0000","last_unstale":"2024-04-01T00:13:48.027240+0000","last_undegraded":"2024-04-01T00:13:48.027240+0000","last_fullsized":"2024-04-01T00:13:48.027240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905298+0000","last_change":"2024-04-01T00:13:46.037508+0000","last_active":"2024-04-01T00:13:48.905298+0000","last_peered":"2024-04-01T00:13:48.905298+0000","last_clean":"2024-04-01T00:13:48.905298+0000","last_became_active":"2024-04-01T00:13:46.037083+0000","last_became_peered":"2024-04-01T00:13:46.037083+0000","last_unstale":"2024-04-01T00:13:48.905298+0000","last_undegraded":"2024-04-01T00:13:48.905298+0000","last_fullsized":"2024-04-01T00:13:48.905298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025575+0000","last_change":"2024-04-01T00:13:47.027434+0000","last_active":"2024-04-01T00:13:48.025575+0000","last_peered":"2024-04-01T00:13:48.025575+0000","last_clean":"2024-04-01T00:13:48.025575+0000","last_became_active":"2024-04-01T00:13:47.027134+0000","last_became_peered":"2024-04-01T00:13:47.027134+0000","last_unstale":"2024-04-01T00:13:48.025575+0000","last_undegraded":"2024-04-01T00:13:48.025575+0000","last_fullsized":"2024-04-01T00:13:48.025575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030010+0000","last_change":"2024-04-01T00:13:46.029352+0000","last_active":"2024-04-01T00:13:48.030010+0000","last_peered":"2024-04-01T00:13:48.030010+0000","last_clean":"2024-04-01T00:13:48.030010+0000","last_became_active":"2024-04-01T00:13:46.029053+0000","last_became_peered":"2024-04-01T00:13:46.029053+0000","last_unstale":"2024-04-01T00:13:48.030010+0000","last_undegraded":"2024-04-01T00:13:48.030010+0000","last_fullsized":"2024-04-01T00:13:48.030010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906349+0000","last_change":"2024-04-01T00:13:47.045641+0000","last_active":"2024-04-01T00:13:48.906349+0000","last_peered":"2024-04-01T00:13:48.906349+0000","last_clean":"2024-04-01T00:13:48.906349+0000","last_became_active":"2024-04-01T00:13:47.045348+0000","last_became_peered":"2024-04-01T00:13:47.045348+0000","last_unstale":"2024-04-01T00:13:48.906349+0000","last_undegraded":"2024-04-01T00:13:48.906349+0000","last_fullsized":"2024-04-01T00:13:48.906349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030805+0000","last_change":"2024-04-01T00:13:46.037865+0000","last_active":"2024-04-01T00:13:48.030805+0000","last_peered":"2024-04-01T00:13:48.030805+0000","last_clean":"2024-04-01T00:13:48.030805+0000","last_became_active":"2024-04-01T00:13:46.037564+0000","last_became_peered":"2024-04-01T00:13:46.037564+0000","last_unstale":"2024-04-01T00:13:48.030805+0000","last_undegraded":"2024-04-01T00:13:48.030805+0000","last_fullsized":"2024-04-01T00:13:48.030805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906532+0000","last_change":"2024-04-01T00:13:47.038745+0000","last_active":"2024-04-01T00:13:48.906532+0000","last_peered":"2024-04-01T00:13:48.906532+0000","last_clean":"2024-04-01T00:13:48.906532+0000","last_became_active":"2024-04-01T00:13:47.038440+0000","last_became_peered":"2024-04-01T00:13:47.038440+0000","last_unstale":"2024-04-01T00:13:48.906532+0000","last_undegraded":"2024-04-01T00:13:48.906532+0000","last_fullsized":"2024-04-01T00:13:48.906532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904810+0000","last_change":"2024-04-01T00:13:46.037408+0000","last_active":"2024-04-01T00:13:48.904810+0000","last_peered":"2024-04-01T00:13:48.904810+0000","last_clean":"2024-04-01T00:13:48.904810+0000","last_became_active":"2024-04-01T00:13:46.037120+0000","last_became_peered":"2024-04-01T00:13:46.037120+0000","last_unstale":"2024-04-01T00:13:48.904810+0000","last_undegraded":"2024-04-01T00:13:48.904810+0000","last_fullsized":"2024-04-01T00:13:48.904810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028332+0000","last_change":"2024-04-01T00:13:47.032590+0000","last_active":"2024-04-01T00:13:48.028332+0000","last_peered":"2024-04-01T00:13:48.028332+0000","last_clean":"2024-04-01T00:13:48.028332+0000","last_became_active":"2024-04-01T00:13:47.032294+0000","last_became_peered":"2024-04-01T00:13:47.032294+0000","last_unstale":"2024-04-01T00:13:48.028332+0000","last_undegraded":"2024-04-01T00:13:48.028332+0000","last_fullsized":"2024-04-01T00:13:48.028332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025536+0000","last_change":"2024-04-01T00:13:46.036386+0000","last_active":"2024-04-01T00:13:48.025536+0000","last_peered":"2024-04-01T00:13:48.025536+0000","last_clean":"2024-04-01T00:13:48.025536+0000","last_became_active":"2024-04-01T00:13:46.036055+0000","last_became_peered":"2024-04-01T00:13:46.036055+0000","last_unstale":"2024-04-01T00:13:48.025536+0000","last_undegraded":"2024-04-01T00:13:48.025536+0000","last_fullsized":"2024-04-01T00:13:48.025536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905628+0000","last_change":"2024-04-01T00:13:47.027936+0000","last_active":"2024-04-01T00:13:48.905628+0000","last_peered":"2024-04-01T00:13:48.905628+0000","last_clean":"2024-04-01T00:13:48.905628+0000","last_became_active":"2024-04-01T00:13:47.027628+0000","last_became_peered":"2024-04-01T00:13:47.027628+0000","last_unstale":"2024-04-01T00:13:48.905628+0000","last_undegraded":"2024-04-01T00:13:48.905628+0000","last_fullsized":"2024-04-01T00:13:48.905628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025431+0000","last_change":"2024-04-01T00:13:46.028586+0000","last_active":"2024-04-01T00:13:48.025431+0000","last_peered":"2024-04-01T00:13:48.025431+0000","last_clean":"2024-04-01T00:13:48.025431+0000","last_became_active":"2024-04-01T00:13:46.028309+0000","last_became_peered":"2024-04-01T00:13:46.028309+0000","last_unstale":"2024-04-01T00:13:48.025431+0000","last_undegraded":"2024-04-01T00:13:48.025431+0000","last_fullsized":"2024-04-01T00:13:48.025431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906125+0000","last_change":"2024-04-01T00:13:47.042379+0000","last_active":"2024-04-01T00:13:48.906125+0000","last_peered":"2024-04-01T00:13:48.906125+0000","last_clean":"2024-04-01T00:13:48.906125+0000","last_became_active":"2024-04-01T00:13:47.042018+0000","last_became_peered":"2024-04-01T00:13:47.042018+0000","last_unstale":"2024-04-01T00:13:48.906125+0000","last_undegraded":"2024-04-01T00:13:48.906125+0000","last_fullsized":"2024-04-01T00:13:48.906125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905276+0000","last_change":"2024-04-01T00:13:46.037179+0000","last_active":"2024-04-01T00:13:48.905276+0000","last_peered":"2024-04-01T00:13:48.905276+0000","last_clean":"2024-04-01T00:13:48.905276+0000","last_became_active":"2024-04-01T00:13:46.036168+0000","last_became_peered":"2024-04-01T00:13:46.036168+0000","last_unstale":"2024-04-01T00:13:48.905276+0000","last_undegraded":"2024-04-01T00:13:48.905276+0000","last_fullsized":"2024-04-01T00:13:48.905276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030533+0000","last_change":"2024-04-01T00:13:47.031588+0000","last_active":"2024-04-01T00:13:48.030533+0000","last_peered":"2024-04-01T00:13:48.030533+0000","last_clean":"2024-04-01T00:13:48.030533+0000","last_became_active":"2024-04-01T00:13:47.031297+0000","last_became_peered":"2024-04-01T00:13:47.031297+0000","last_unstale":"2024-04-01T00:13:48.030533+0000","last_undegraded":"2024-04-01T00:13:48.030533+0000","last_fullsized":"2024-04-01T00:13:48.030533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027184+0000","last_change":"2024-04-01T00:13:47.036405+0000","last_active":"2024-04-01T00:13:48.027184+0000","last_peered":"2024-04-01T00:13:48.027184+0000","last_clean":"2024-04-01T00:13:48.027184+0000","last_became_active":"2024-04-01T00:13:47.036144+0000","last_became_peered":"2024-04-01T00:13:47.036144+0000","last_unstale":"2024-04-01T00:13:48.027184+0000","last_undegraded":"2024-04-01T00:13:48.027184+0000","last_fullsized":"2024-04-01T00:13:48.027184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905062+0000","last_change":"2024-04-01T00:13:46.036339+0000","last_active":"2024-04-01T00:13:48.905062+0000","last_peered":"2024-04-01T00:13:48.905062+0000","last_clean":"2024-04-01T00:13:48.905062+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.905062+0000","last_undegraded":"2024-04-01T00:13:48.905062+0000","last_fullsized":"2024-04-01T00:13:48.905062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:50.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026918+0000","last_change":"2024-04-01T00:13:47.037119+0000","last_active":"2024-04-01T00:13:48.026918+0000","last_peered":"2024-04-01T00:13:48.026918+0000","last_clean":"2024-04-01T00:13:48.026918+0000","last_became_active":"2024-04-01T00:13:47.036860+0000","last_became_peered":"2024-04-01T00:13:47.036860+0000","last_unstale":"2024-04-01T00:13:48.026918+0000","last_undegraded":"2024-04-01T00:13:48.026918+0000","last_fullsized":"2024-04-01T00:13:48.026918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905203+0000","last_change":"2024-04-01T00:13:46.036988+0000","last_active":"2024-04-01T00:13:48.905203+0000","last_peered":"2024-04-01T00:13:48.905203+0000","last_clean":"2024-04-01T00:13:48.905203+0000","last_became_active":"2024-04-01T00:13:46.036696+0000","last_became_peered":"2024-04-01T00:13:46.036696+0000","last_unstale":"2024-04-01T00:13:48.905203+0000","last_undegraded":"2024-04-01T00:13:48.905203+0000","last_fullsized":"2024-04-01T00:13:48.905203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906528+0000","last_change":"2024-04-01T00:13:47.041150+0000","last_active":"2024-04-01T00:13:48.906528+0000","last_peered":"2024-04-01T00:13:48.906528+0000","last_clean":"2024-04-01T00:13:48.906528+0000","last_became_active":"2024-04-01T00:13:47.040826+0000","last_became_peered":"2024-04-01T00:13:47.040826+0000","last_unstale":"2024-04-01T00:13:48.906528+0000","last_undegraded":"2024-04-01T00:13:48.906528+0000","last_fullsized":"2024-04-01T00:13:48.906528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906154+0000","last_change":"2024-04-01T00:13:46.032149+0000","last_active":"2024-04-01T00:13:48.906154+0000","last_peered":"2024-04-01T00:13:48.906154+0000","last_clean":"2024-04-01T00:13:48.906154+0000","last_became_active":"2024-04-01T00:13:46.031821+0000","last_became_peered":"2024-04-01T00:13:46.031821+0000","last_unstale":"2024-04-01T00:13:48.906154+0000","last_undegraded":"2024-04-01T00:13:48.906154+0000","last_fullsized":"2024-04-01T00:13:48.906154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905922+0000","last_change":"2024-04-01T00:13:47.031502+0000","last_active":"2024-04-01T00:13:48.905922+0000","last_peered":"2024-04-01T00:13:48.905922+0000","last_clean":"2024-04-01T00:13:48.905922+0000","last_became_active":"2024-04-01T00:13:47.030468+0000","last_became_peered":"2024-04-01T00:13:47.030468+0000","last_unstale":"2024-04-01T00:13:48.905922+0000","last_undegraded":"2024-04-01T00:13:48.905922+0000","last_fullsized":"2024-04-01T00:13:48.905922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906670+0000","last_change":"2024-04-01T00:13:46.038424+0000","last_active":"2024-04-01T00:13:48.906670+0000","last_peered":"2024-04-01T00:13:48.906670+0000","last_clean":"2024-04-01T00:13:48.906670+0000","last_became_active":"2024-04-01T00:13:46.038079+0000","last_became_peered":"2024-04-01T00:13:46.038079+0000","last_unstale":"2024-04-01T00:13:48.906670+0000","last_undegraded":"2024-04-01T00:13:48.906670+0000","last_fullsized":"2024-04-01T00:13:48.906670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:44:48.269461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:57.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:13:57.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.605+0000 7f7fa8007640 1 Processor -- start 2024-04-01T00:13:57.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.605+0000 7f7fa8007640 1 -- start start 2024-04-01T00:13:57.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa8007640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7fa015e1c0 0x7f7fa00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa8007640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fa006caa0 0x7f7fa00d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa8007640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa00d0060 con 0x7f7fa00cc340 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa8007640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa00d0650 con 0x7f7fa015e1c0 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa8007640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa006d100 con 0x7f7fa006caa0 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7fa015e1c0 0x7f7fa00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7fa015e1c0 0x7f7fa00d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51348/0 (socket says 172.21.15.64:51348) 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 -- 172.21.15.64:0/873270387 learned_addr learned my addr 172.21.15.64:0/873270387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa65a4640 1 --2- 172.21.15.64:0/873270387 >> v2:172.21.15.100:3300/0 conn(0x7f7fa006caa0 0x7f7fa00d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 -- 172.21.15.64:0/873270387 >> v2:172.21.15.100:3300/0 conn(0x7f7fa006caa0 msgr2=0x7f7fa00d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 --2- 172.21.15.64:0/873270387 >> v2:172.21.15.100:3300/0 conn(0x7f7fa006caa0 0x7f7fa00d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 -- 172.21.15.64:0/873270387 >> v1:172.21.15.100:6789/0 conn(0x7f7fa00cc340 legacy=0x7f7fa006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.606+0000 7f7fa5da3640 1 -- 172.21.15.64:0/873270387 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa006d3e0 con 0x7f7fa015e1c0 2024-04-01T00:13:57.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa5da3640 1 --2- 172.21.15.64:0/873270387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa015e1c0 0x7f7fa00d1c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b2030 tx=0x7f7f9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396784b190d3edfd server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa4da1640 1 -- 172.21.15.64:0/873270387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c035020 con 0x7f7fa015e1c0 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa4da1640 1 -- 172.21.15.64:0/873270387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c031870 con 0x7f7fa015e1c0 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa4da1640 1 -- 172.21.15.64:0/873270387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c031c40 con 0x7f7fa015e1c0 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 -- 172.21.15.64:0/873270387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa015e1c0 msgr2=0x7f7fa00d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 --2- 172.21.15.64:0/873270387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa015e1c0 0x7f7fa00d1c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b2030 tx=0x7f7f9c02b910 comp rx=0 tx=0).stop 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 -- 172.21.15.64:0/873270387 shutdown_connections 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 --2- 172.21.15.64:0/873270387 >> v2:172.21.15.100:3300/0 conn(0x7f7fa006caa0 0x7f7fa00d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 --2- 172.21.15.64:0/873270387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa015e1c0 0x7f7fa00d1c10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.607+0000 7f7fa8007640 1 -- 172.21.15.64:0/873270387 >> 172.21.15.64:0/873270387 conn(0x7f7fa009a980 msgr2=0x7f7fa009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.608+0000 7f7fa8007640 1 -- 172.21.15.64:0/873270387 shutdown_connections 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.608+0000 7f7fa8007640 1 -- 172.21.15.64:0/873270387 wait complete. 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 Processor -- start 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 -- start start 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00cc340 0x7f7fa00cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015e1c0 0x7f7fa0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa00b0b80 con 0x7f7fa006caa0 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa00b0a00 con 0x7f7fa00cc340 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.609+0000 7f7fa8007640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa00b0d00 con 0x7f7fa015e1c0 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51360/0 (socket says 172.21.15.64:51360) 2024-04-01T00:13:57.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 -- 172.21.15.64:0/3385428807 learned_addr learned my addr 172.21.15.64:0/3385428807 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa55a2640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00cc340 0x7f7fa00cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa65a4640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015e1c0 0x7f7fa0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 -- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015e1c0 msgr2=0x7f7fa0153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015e1c0 0x7f7fa0153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 -- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00cc340 msgr2=0x7f7fa00cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00cc340 0x7f7fa00cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 -- 172.21.15.64:0/3385428807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa015c5c0 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa5da3640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c02b950 tx=0x7f7f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c040070 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c030d00 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa00c9d90 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c035040 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.610+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa00ca350 con 0x7f7fa006caa0 2024-04-01T00:13:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.611+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa00681a0 con 0x7f7fa006caa0 2024-04-01T00:13:57.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.612+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7f9c038020 con 0x7f7fa006caa0 2024-04-01T00:13:57.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.612+0000 7f7f967fc640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 0x7f7f8406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.612+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f9c0ac4f0 con 0x7f7fa006caa0 2024-04-01T00:13:57.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.612+0000 7f7fa55a2640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 0x7f7f8406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.613+0000 7f7fa55a2640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 0x7f7f8406de50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7fa01f7010 tx=0x7f7f90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.617+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f7fa015c5c0 con 0x7f7fa006caa0 2024-04-01T00:13:57.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.796+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7fa009b170 con 0x7f7f8406b990 2024-04-01T00:13:57.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.802+0000 7f7f967fc640 1 -- 172.21.15.64:0/3385428807 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f7fa009b170 con 0x7f7f8406b990 2024-04-01T00:13:57.806 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:13:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 msgr2=0x7f7f8406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 0x7f7f8406de50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7fa01f7010 tx=0x7f7f90009040 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 msgr2=0x7f7fa00b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c02b950 tx=0x7f7f9c0046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 shutdown_connections 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7f8406b990 0x7f7f8406de50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015e1c0 0x7f7fa0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00cc340 0x7f7fa00cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 --2- 172.21.15.64:0/3385428807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fa006caa0 0x7f7fa00b7440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 >> 172.21.15.64:0/3385428807 conn(0x7f7fa009a980 msgr2=0x7f7fa00cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 shutdown_connections 2024-04-01T00:13:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.805+0000 7f7fa8007640 1 -- 172.21.15.64:0/3385428807 wait complete. 2024-04-01T00:13:57.817 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-01T00:13:56.747821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.731501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905422+0000","last_change":"2024-04-01T00:13:47.045277+0000","last_active":"2024-04-01T00:13:48.905422+0000","last_peered":"2024-04-01T00:13:48.905422+0000","last_clean":"2024-04-01T00:13:48.905422+0000","last_became_active":"2024-04-01T00:13:47.044728+0000","last_became_peered":"2024-04-01T00:13:47.044728+0000","last_unstale":"2024-04-01T00:13:48.905422+0000","last_undegraded":"2024-04-01T00:13:48.905422+0000","last_fullsized":"2024-04-01T00:13:48.905422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:50:54.919900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025542+0000","last_change":"2024-04-01T00:13:47.039324+0000","last_active":"2024-04-01T00:13:48.025542+0000","last_peered":"2024-04-01T00:13:48.025542+0000","last_clean":"2024-04-01T00:13:48.025542+0000","last_became_active":"2024-04-01T00:13:47.039002+0000","last_became_peered":"2024-04-01T00:13:47.039002+0000","last_unstale":"2024-04-01T00:13:48.025542+0000","last_undegraded":"2024-04-01T00:13:48.025542+0000","last_fullsized":"2024-04-01T00:13:48.025542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906198+0000","last_change":"2024-04-01T00:13:47.039670+0000","last_active":"2024-04-01T00:13:48.906198+0000","last_peered":"2024-04-01T00:13:48.906198+0000","last_clean":"2024-04-01T00:13:48.906198+0000","last_became_active":"2024-04-01T00:13:47.037096+0000","last_became_peered":"2024-04-01T00:13:47.037096+0000","last_unstale":"2024-04-01T00:13:48.906198+0000","last_undegraded":"2024-04-01T00:13:48.906198+0000","last_fullsized":"2024-04-01T00:13:48.906198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:05:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026858+0000","last_change":"2024-04-01T00:13:47.033997+0000","last_active":"2024-04-01T00:13:48.026858+0000","last_peered":"2024-04-01T00:13:48.026858+0000","last_clean":"2024-04-01T00:13:48.026858+0000","last_became_active":"2024-04-01T00:13:47.033682+0000","last_became_peered":"2024-04-01T00:13:47.033682+0000","last_unstale":"2024-04-01T00:13:48.026858+0000","last_undegraded":"2024-04-01T00:13:48.026858+0000","last_fullsized":"2024-04-01T00:13:48.026858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905169+0000","last_change":"2024-04-01T00:13:47.040873+0000","last_active":"2024-04-01T00:13:48.905169+0000","last_peered":"2024-04-01T00:13:48.905169+0000","last_clean":"2024-04-01T00:13:48.905169+0000","last_became_active":"2024-04-01T00:13:47.039953+0000","last_became_peered":"2024-04-01T00:13:47.039953+0000","last_unstale":"2024-04-01T00:13:48.905169+0000","last_undegraded":"2024-04-01T00:13:48.905169+0000","last_fullsized":"2024-04-01T00:13:48.905169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026638+0000","last_change":"2024-04-01T00:13:47.041029+0000","last_active":"2024-04-01T00:13:48.026638+0000","last_peered":"2024-04-01T00:13:48.026638+0000","last_clean":"2024-04-01T00:13:48.026638+0000","last_became_active":"2024-04-01T00:13:47.040771+0000","last_became_peered":"2024-04-01T00:13:47.040771+0000","last_unstale":"2024-04-01T00:13:48.026638+0000","last_undegraded":"2024-04-01T00:13:48.026638+0000","last_fullsized":"2024-04-01T00:13:48.026638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904941+0000","last_change":"2024-04-01T00:13:47.042649+0000","last_active":"2024-04-01T00:13:48.904941+0000","last_peered":"2024-04-01T00:13:48.904941+0000","last_clean":"2024-04-01T00:13:48.904941+0000","last_became_active":"2024-04-01T00:13:47.042373+0000","last_became_peered":"2024-04-01T00:13:47.042373+0000","last_unstale":"2024-04-01T00:13:48.904941+0000","last_undegraded":"2024-04-01T00:13:48.904941+0000","last_fullsized":"2024-04-01T00:13:48.904941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905488+0000","last_change":"2024-04-01T00:13:47.031258+0000","last_active":"2024-04-01T00:13:48.905488+0000","last_peered":"2024-04-01T00:13:48.905488+0000","last_clean":"2024-04-01T00:13:48.905488+0000","last_became_active":"2024-04-01T00:13:47.030972+0000","last_became_peered":"2024-04-01T00:13:47.030972+0000","last_unstale":"2024-04-01T00:13:48.905488+0000","last_undegraded":"2024-04-01T00:13:48.905488+0000","last_fullsized":"2024-04-01T00:13:48.905488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905452+0000","last_change":"2024-04-01T00:13:47.045417+0000","last_active":"2024-04-01T00:13:48.905452+0000","last_peered":"2024-04-01T00:13:48.905452+0000","last_clean":"2024-04-01T00:13:48.905452+0000","last_became_active":"2024-04-01T00:13:47.044978+0000","last_became_peered":"2024-04-01T00:13:47.044978+0000","last_unstale":"2024-04-01T00:13:48.905452+0000","last_undegraded":"2024-04-01T00:13:48.905452+0000","last_fullsized":"2024-04-01T00:13:48.905452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:12:06.404460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030825+0000","last_change":"2024-04-01T00:13:47.035602+0000","last_active":"2024-04-01T00:13:48.030825+0000","last_peered":"2024-04-01T00:13:48.030825+0000","last_clean":"2024-04-01T00:13:48.030825+0000","last_became_active":"2024-04-01T00:13:47.035293+0000","last_became_peered":"2024-04-01T00:13:47.035293+0000","last_unstale":"2024-04-01T00:13:48.030825+0000","last_undegraded":"2024-04-01T00:13:48.030825+0000","last_fullsized":"2024-04-01T00:13:48.030825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030712+0000","last_change":"2024-04-01T00:13:47.037792+0000","last_active":"2024-04-01T00:13:48.030712+0000","last_peered":"2024-04-01T00:13:48.030712+0000","last_clean":"2024-04-01T00:13:48.030712+0000","last_became_active":"2024-04-01T00:13:47.037508+0000","last_became_peered":"2024-04-01T00:13:47.037508+0000","last_unstale":"2024-04-01T00:13:48.030712+0000","last_undegraded":"2024-04-01T00:13:48.030712+0000","last_fullsized":"2024-04-01T00:13:48.030712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028123+0000","last_change":"2024-04-01T00:13:47.026838+0000","last_active":"2024-04-01T00:13:48.028123+0000","last_peered":"2024-04-01T00:13:48.028123+0000","last_clean":"2024-04-01T00:13:48.028123+0000","last_became_active":"2024-04-01T00:13:47.026586+0000","last_became_peered":"2024-04-01T00:13:47.026586+0000","last_unstale":"2024-04-01T00:13:48.028123+0000","last_undegraded":"2024-04-01T00:13:48.028123+0000","last_fullsized":"2024-04-01T00:13:48.028123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:13.513798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905964+0000","last_change":"2024-04-01T00:13:47.028848+0000","last_active":"2024-04-01T00:13:48.905964+0000","last_peered":"2024-04-01T00:13:48.905964+0000","last_clean":"2024-04-01T00:13:48.905964+0000","last_became_active":"2024-04-01T00:13:47.028577+0000","last_became_peered":"2024-04-01T00:13:47.028577+0000","last_unstale":"2024-04-01T00:13:48.905964+0000","last_undegraded":"2024-04-01T00:13:48.905964+0000","last_fullsized":"2024-04-01T00:13:48.905964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906018+0000","last_change":"2024-04-01T00:13:47.039847+0000","last_active":"2024-04-01T00:13:48.906018+0000","last_peered":"2024-04-01T00:13:48.906018+0000","last_clean":"2024-04-01T00:13:48.906018+0000","last_became_active":"2024-04-01T00:13:47.039459+0000","last_became_peered":"2024-04-01T00:13:47.039459+0000","last_unstale":"2024-04-01T00:13:48.906018+0000","last_undegraded":"2024-04-01T00:13:48.906018+0000","last_fullsized":"2024-04-01T00:13:48.906018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:51.654019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025413+0000","last_change":"2024-04-01T00:13:47.034288+0000","last_active":"2024-04-01T00:13:48.025413+0000","last_peered":"2024-04-01T00:13:48.025413+0000","last_clean":"2024-04-01T00:13:48.025413+0000","last_became_active":"2024-04-01T00:13:47.034022+0000","last_became_peered":"2024-04-01T00:13:47.034022+0000","last_unstale":"2024-04-01T00:13:48.025413+0000","last_undegraded":"2024-04-01T00:13:48.025413+0000","last_fullsized":"2024-04-01T00:13:48.025413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026557+0000","last_change":"2024-04-01T00:13:47.039173+0000","last_active":"2024-04-01T00:13:48.026557+0000","last_peered":"2024-04-01T00:13:48.026557+0000","last_clean":"2024-04-01T00:13:48.026557+0000","last_became_active":"2024-04-01T00:13:47.038845+0000","last_became_peered":"2024-04-01T00:13:47.038845+0000","last_unstale":"2024-04-01T00:13:48.026557+0000","last_undegraded":"2024-04-01T00:13:48.026557+0000","last_fullsized":"2024-04-01T00:13:48.026557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027411+0000","last_change":"2024-04-01T00:13:47.043400+0000","last_active":"2024-04-01T00:13:48.027411+0000","last_peered":"2024-04-01T00:13:48.027411+0000","last_clean":"2024-04-01T00:13:48.027411+0000","last_became_active":"2024-04-01T00:13:47.043120+0000","last_became_peered":"2024-04-01T00:13:47.043120+0000","last_unstale":"2024-04-01T00:13:48.027411+0000","last_undegraded":"2024-04-01T00:13:48.027411+0000","last_fullsized":"2024-04-01T00:13:48.027411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905429+0000","last_change":"2024-04-01T00:13:47.035436+0000","last_active":"2024-04-01T00:13:48.905429+0000","last_peered":"2024-04-01T00:13:48.905429+0000","last_clean":"2024-04-01T00:13:48.905429+0000","last_became_active":"2024-04-01T00:13:47.035072+0000","last_became_peered":"2024-04-01T00:13:47.035072+0000","last_unstale":"2024-04-01T00:13:48.905429+0000","last_undegraded":"2024-04-01T00:13:48.905429+0000","last_fullsized":"2024-04-01T00:13:48.905429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905379+0000","last_change":"2024-04-01T00:13:47.040506+0000","last_active":"2024-04-01T00:13:48.905379+0000","last_peered":"2024-04-01T00:13:48.905379+0000","last_clean":"2024-04-01T00:13:48.905379+0000","last_became_active":"2024-04-01T00:13:47.040244+0000","last_became_peered":"2024-04-01T00:13:47.040244+0000","last_unstale":"2024-04-01T00:13:48.905379+0000","last_undegraded":"2024-04-01T00:13:48.905379+0000","last_fullsized":"2024-04-01T00:13:48.905379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:09:41.839861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030707+0000","last_change":"2024-04-01T00:13:47.031532+0000","last_active":"2024-04-01T00:13:48.030707+0000","last_peered":"2024-04-01T00:13:48.030707+0000","last_clean":"2024-04-01T00:13:48.030707+0000","last_became_active":"2024-04-01T00:13:47.031252+0000","last_became_peered":"2024-04-01T00:13:47.031252+0000","last_unstale":"2024-04-01T00:13:48.030707+0000","last_undegraded":"2024-04-01T00:13:48.030707+0000","last_fullsized":"2024-04-01T00:13:48.030707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906466+0000","last_change":"2024-04-01T00:13:47.047404+0000","last_active":"2024-04-01T00:13:48.906466+0000","last_peered":"2024-04-01T00:13:48.906466+0000","last_clean":"2024-04-01T00:13:48.906466+0000","last_became_active":"2024-04-01T00:13:47.047080+0000","last_became_peered":"2024-04-01T00:13:47.047080+0000","last_unstale":"2024-04-01T00:13:48.906466+0000","last_undegraded":"2024-04-01T00:13:48.906466+0000","last_fullsized":"2024-04-01T00:13:48.906466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026698+0000","last_change":"2024-04-01T00:13:47.033044+0000","last_active":"2024-04-01T00:13:48.026698+0000","last_peered":"2024-04-01T00:13:48.026698+0000","last_clean":"2024-04-01T00:13:48.026698+0000","last_became_active":"2024-04-01T00:13:47.032783+0000","last_became_peered":"2024-04-01T00:13:47.032783+0000","last_unstale":"2024-04-01T00:13:48.026698+0000","last_undegraded":"2024-04-01T00:13:48.026698+0000","last_fullsized":"2024-04-01T00:13:48.026698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905485+0000","last_change":"2024-04-01T00:13:47.035282+0000","last_active":"2024-04-01T00:13:48.905485+0000","last_peered":"2024-04-01T00:13:48.905485+0000","last_clean":"2024-04-01T00:13:48.905485+0000","last_became_active":"2024-04-01T00:13:47.034953+0000","last_became_peered":"2024-04-01T00:13:47.034953+0000","last_unstale":"2024-04-01T00:13:48.905485+0000","last_undegraded":"2024-04-01T00:13:48.905485+0000","last_fullsized":"2024-04-01T00:13:48.905485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026661+0000","last_change":"2024-04-01T00:13:47.034177+0000","last_active":"2024-04-01T00:13:48.026661+0000","last_peered":"2024-04-01T00:13:48.026661+0000","last_clean":"2024-04-01T00:13:48.026661+0000","last_became_active":"2024-04-01T00:13:47.033426+0000","last_became_peered":"2024-04-01T00:13:47.033426+0000","last_unstale":"2024-04-01T00:13:48.026661+0000","last_undegraded":"2024-04-01T00:13:48.026661+0000","last_fullsized":"2024-04-01T00:13:48.026661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906196+0000","last_change":"2024-04-01T00:13:47.034202+0000","last_active":"2024-04-01T00:13:48.906196+0000","last_peered":"2024-04-01T00:13:48.906196+0000","last_clean":"2024-04-01T00:13:48.906196+0000","last_became_active":"2024-04-01T00:13:47.033941+0000","last_became_peered":"2024-04-01T00:13:47.033941+0000","last_unstale":"2024-04-01T00:13:48.906196+0000","last_undegraded":"2024-04-01T00:13:48.906196+0000","last_fullsized":"2024-04-01T00:13:48.906196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030886+0000","last_change":"2024-04-01T00:13:47.042844+0000","last_active":"2024-04-01T00:13:48.030886+0000","last_peered":"2024-04-01T00:13:48.030886+0000","last_clean":"2024-04-01T00:13:48.030886+0000","last_became_active":"2024-04-01T00:13:47.042518+0000","last_became_peered":"2024-04-01T00:13:47.042518+0000","last_unstale":"2024-04-01T00:13:48.030886+0000","last_undegraded":"2024-04-01T00:13:48.030886+0000","last_fullsized":"2024-04-01T00:13:48.030886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025478+0000","last_change":"2024-04-01T00:13:47.032968+0000","last_active":"2024-04-01T00:13:48.025478+0000","last_peered":"2024-04-01T00:13:48.025478+0000","last_clean":"2024-04-01T00:13:48.025478+0000","last_became_active":"2024-04-01T00:13:47.032695+0000","last_became_peered":"2024-04-01T00:13:47.032695+0000","last_unstale":"2024-04-01T00:13:48.025478+0000","last_undegraded":"2024-04-01T00:13:48.025478+0000","last_fullsized":"2024-04-01T00:13:48.025478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906706+0000","last_change":"2024-04-01T00:13:47.035447+0000","last_active":"2024-04-01T00:13:48.906706+0000","last_peered":"2024-04-01T00:13:48.906706+0000","last_clean":"2024-04-01T00:13:48.906706+0000","last_became_active":"2024-04-01T00:13:47.035003+0000","last_became_peered":"2024-04-01T00:13:47.035003+0000","last_unstale":"2024-04-01T00:13:48.906706+0000","last_undegraded":"2024-04-01T00:13:48.906706+0000","last_fullsized":"2024-04-01T00:13:48.906706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906541+0000","last_change":"2024-04-01T00:13:47.039218+0000","last_active":"2024-04-01T00:13:48.906541+0000","last_peered":"2024-04-01T00:13:48.906541+0000","last_clean":"2024-04-01T00:13:48.906541+0000","last_became_active":"2024-04-01T00:13:47.037984+0000","last_became_peered":"2024-04-01T00:13:47.037984+0000","last_unstale":"2024-04-01T00:13:48.906541+0000","last_undegraded":"2024-04-01T00:13:48.906541+0000","last_fullsized":"2024-04-01T00:13:48.906541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026792+0000","last_change":"2024-04-01T00:13:47.034521+0000","last_active":"2024-04-01T00:13:48.026792+0000","last_peered":"2024-04-01T00:13:48.026792+0000","last_clean":"2024-04-01T00:13:48.026792+0000","last_became_active":"2024-04-01T00:13:47.034251+0000","last_became_peered":"2024-04-01T00:13:47.034251+0000","last_unstale":"2024-04-01T00:13:48.026792+0000","last_undegraded":"2024-04-01T00:13:48.026792+0000","last_fullsized":"2024-04-01T00:13:48.026792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905386+0000","last_change":"2024-04-01T00:13:47.035817+0000","last_active":"2024-04-01T00:13:48.905386+0000","last_peered":"2024-04-01T00:13:48.905386+0000","last_clean":"2024-04-01T00:13:48.905386+0000","last_became_active":"2024-04-01T00:13:47.034834+0000","last_became_peered":"2024-04-01T00:13:47.034834+0000","last_unstale":"2024-04-01T00:13:48.905386+0000","last_undegraded":"2024-04-01T00:13:48.905386+0000","last_fullsized":"2024-04-01T00:13:48.905386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030497+0000","last_change":"2024-04-01T00:13:47.039829+0000","last_active":"2024-04-01T00:13:48.030497+0000","last_peered":"2024-04-01T00:13:48.030497+0000","last_clean":"2024-04-01T00:13:48.030497+0000","last_became_active":"2024-04-01T00:13:47.039517+0000","last_became_peered":"2024-04-01T00:13:47.039517+0000","last_unstale":"2024-04-01T00:13:48.030497+0000","last_undegraded":"2024-04-01T00:13:48.030497+0000","last_fullsized":"2024-04-01T00:13:48.030497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027883+0000","last_change":"2024-04-01T00:13:47.026124+0000","last_active":"2024-04-01T00:13:48.027883+0000","last_peered":"2024-04-01T00:13:48.027883+0000","last_clean":"2024-04-01T00:13:48.027883+0000","last_became_active":"2024-04-01T00:13:47.025855+0000","last_became_peered":"2024-04-01T00:13:47.025855+0000","last_unstale":"2024-04-01T00:13:48.027883+0000","last_undegraded":"2024-04-01T00:13:48.027883+0000","last_fullsized":"2024-04-01T00:13:48.027883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:56.178747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906229+0000","last_change":"2024-04-01T00:13:46.037694+0000","last_active":"2024-04-01T00:13:48.906229+0000","last_peered":"2024-04-01T00:13:48.906229+0000","last_clean":"2024-04-01T00:13:48.906229+0000","last_became_active":"2024-04-01T00:13:46.037390+0000","last_became_peered":"2024-04-01T00:13:46.037390+0000","last_unstale":"2024-04-01T00:13:48.906229+0000","last_undegraded":"2024-04-01T00:13:48.906229+0000","last_fullsized":"2024-04-01T00:13:48.906229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906550+0000","last_change":"2024-04-01T00:13:47.044544+0000","last_active":"2024-04-01T00:13:48.906550+0000","last_peered":"2024-04-01T00:13:48.906550+0000","last_clean":"2024-04-01T00:13:48.906550+0000","last_became_active":"2024-04-01T00:13:47.044266+0000","last_became_peered":"2024-04-01T00:13:47.044266+0000","last_unstale":"2024-04-01T00:13:48.906550+0000","last_undegraded":"2024-04-01T00:13:48.906550+0000","last_fullsized":"2024-04-01T00:13:48.906550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030621+0000","last_change":"2024-04-01T00:13:46.036332+0000","last_active":"2024-04-01T00:13:48.030621+0000","last_peered":"2024-04-01T00:13:48.030621+0000","last_clean":"2024-04-01T00:13:48.030621+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.030621+0000","last_undegraded":"2024-04-01T00:13:48.030621+0000","last_fullsized":"2024-04-01T00:13:48.030621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905834+0000","last_change":"2024-04-01T00:13:46.032078+0000","last_active":"2024-04-01T00:13:48.905834+0000","last_peered":"2024-04-01T00:13:48.905834+0000","last_clean":"2024-04-01T00:13:48.905834+0000","last_became_active":"2024-04-01T00:13:46.031705+0000","last_became_peered":"2024-04-01T00:13:46.031705+0000","last_unstale":"2024-04-01T00:13:48.905834+0000","last_undegraded":"2024-04-01T00:13:48.905834+0000","last_fullsized":"2024-04-01T00:13:48.905834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905625+0000","last_change":"2024-04-01T00:13:47.041444+0000","last_active":"2024-04-01T00:13:48.905625+0000","last_peered":"2024-04-01T00:13:48.905625+0000","last_clean":"2024-04-01T00:13:48.905625+0000","last_became_active":"2024-04-01T00:13:47.041163+0000","last_became_peered":"2024-04-01T00:13:47.041163+0000","last_unstale":"2024-04-01T00:13:48.905625+0000","last_undegraded":"2024-04-01T00:13:48.905625+0000","last_fullsized":"2024-04-01T00:13:48.905625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026649+0000","last_change":"2024-04-01T00:13:46.030376+0000","last_active":"2024-04-01T00:13:48.026649+0000","last_peered":"2024-04-01T00:13:48.026649+0000","last_clean":"2024-04-01T00:13:48.026649+0000","last_became_active":"2024-04-01T00:13:46.030085+0000","last_became_peered":"2024-04-01T00:13:46.030085+0000","last_unstale":"2024-04-01T00:13:48.026649+0000","last_undegraded":"2024-04-01T00:13:48.026649+0000","last_fullsized":"2024-04-01T00:13:48.026649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025414+0000","last_change":"2024-04-01T00:13:47.032819+0000","last_active":"2024-04-01T00:13:48.025414+0000","last_peered":"2024-04-01T00:13:48.025414+0000","last_clean":"2024-04-01T00:13:48.025414+0000","last_became_active":"2024-04-01T00:13:47.032501+0000","last_became_peered":"2024-04-01T00:13:47.032501+0000","last_unstale":"2024-04-01T00:13:48.025414+0000","last_undegraded":"2024-04-01T00:13:48.025414+0000","last_fullsized":"2024-04-01T00:13:48.025414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:12:44.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030703+0000","last_change":"2024-04-01T00:13:46.030622+0000","last_active":"2024-04-01T00:13:48.030703+0000","last_peered":"2024-04-01T00:13:48.030703+0000","last_clean":"2024-04-01T00:13:48.030703+0000","last_became_active":"2024-04-01T00:13:46.030340+0000","last_became_peered":"2024-04-01T00:13:46.030340+0000","last_unstale":"2024-04-01T00:13:48.030703+0000","last_undegraded":"2024-04-01T00:13:48.030703+0000","last_fullsized":"2024-04-01T00:13:48.030703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027187+0000","last_change":"2024-04-01T00:13:47.042195+0000","last_active":"2024-04-01T00:13:48.027187+0000","last_peered":"2024-04-01T00:13:48.027187+0000","last_clean":"2024-04-01T00:13:48.027187+0000","last_became_active":"2024-04-01T00:13:47.041925+0000","last_became_peered":"2024-04-01T00:13:47.041925+0000","last_unstale":"2024-04-01T00:13:48.027187+0000","last_undegraded":"2024-04-01T00:13:48.027187+0000","last_fullsized":"2024-04-01T00:13:48.027187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:53:05.544053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027178+0000","last_change":"2024-04-01T00:13:46.029157+0000","last_active":"2024-04-01T00:13:48.027178+0000","last_peered":"2024-04-01T00:13:48.027178+0000","last_clean":"2024-04-01T00:13:48.027178+0000","last_became_active":"2024-04-01T00:13:46.028917+0000","last_became_peered":"2024-04-01T00:13:46.028917+0000","last_unstale":"2024-04-01T00:13:48.027178+0000","last_undegraded":"2024-04-01T00:13:48.027178+0000","last_fullsized":"2024-04-01T00:13:48.027178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906652+0000","last_change":"2024-04-01T00:13:47.040888+0000","last_active":"2024-04-01T00:13:48.906652+0000","last_peered":"2024-04-01T00:13:48.906652+0000","last_clean":"2024-04-01T00:13:48.906652+0000","last_became_active":"2024-04-01T00:13:47.040621+0000","last_became_peered":"2024-04-01T00:13:47.040621+0000","last_unstale":"2024-04-01T00:13:48.906652+0000","last_undegraded":"2024-04-01T00:13:48.906652+0000","last_fullsized":"2024-04-01T00:13:48.906652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025420+0000","last_change":"2024-04-01T00:13:46.030545+0000","last_active":"2024-04-01T00:13:48.025420+0000","last_peered":"2024-04-01T00:13:48.025420+0000","last_clean":"2024-04-01T00:13:48.025420+0000","last_became_active":"2024-04-01T00:13:46.030232+0000","last_became_peered":"2024-04-01T00:13:46.030232+0000","last_unstale":"2024-04-01T00:13:48.025420+0000","last_undegraded":"2024-04-01T00:13:48.025420+0000","last_fullsized":"2024-04-01T00:13:48.025420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028260+0000","last_change":"2024-04-01T00:13:47.042059+0000","last_active":"2024-04-01T00:13:48.028260+0000","last_peered":"2024-04-01T00:13:48.028260+0000","last_clean":"2024-04-01T00:13:48.028260+0000","last_became_active":"2024-04-01T00:13:47.040047+0000","last_became_peered":"2024-04-01T00:13:47.040047+0000","last_unstale":"2024-04-01T00:13:48.028260+0000","last_undegraded":"2024-04-01T00:13:48.028260+0000","last_fullsized":"2024-04-01T00:13:48.028260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:20.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030779+0000","last_change":"2024-04-01T00:13:46.039366+0000","last_active":"2024-04-01T00:13:48.030779+0000","last_peered":"2024-04-01T00:13:48.030779+0000","last_clean":"2024-04-01T00:13:48.030779+0000","last_became_active":"2024-04-01T00:13:46.039132+0000","last_became_peered":"2024-04-01T00:13:46.039132+0000","last_unstale":"2024-04-01T00:13:48.030779+0000","last_undegraded":"2024-04-01T00:13:48.030779+0000","last_fullsized":"2024-04-01T00:13:48.030779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905125+0000","last_change":"2024-04-01T00:13:47.039118+0000","last_active":"2024-04-01T00:13:48.905125+0000","last_peered":"2024-04-01T00:13:48.905125+0000","last_clean":"2024-04-01T00:13:48.905125+0000","last_became_active":"2024-04-01T00:13:47.035193+0000","last_became_peered":"2024-04-01T00:13:47.035193+0000","last_unstale":"2024-04-01T00:13:48.905125+0000","last_undegraded":"2024-04-01T00:13:48.905125+0000","last_fullsized":"2024-04-01T00:13:48.905125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906606+0000","last_change":"2024-04-01T00:13:46.033537+0000","last_active":"2024-04-01T00:13:48.906606+0000","last_peered":"2024-04-01T00:13:48.906606+0000","last_clean":"2024-04-01T00:13:48.906606+0000","last_became_active":"2024-04-01T00:13:46.033174+0000","last_became_peered":"2024-04-01T00:13:46.033174+0000","last_unstale":"2024-04-01T00:13:48.906606+0000","last_undegraded":"2024-04-01T00:13:48.906606+0000","last_fullsized":"2024-04-01T00:13:48.906606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:15:18.515585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906163+0000","last_change":"2024-04-01T00:13:47.046668+0000","last_active":"2024-04-01T00:13:48.906163+0000","last_peered":"2024-04-01T00:13:48.906163+0000","last_clean":"2024-04-01T00:13:48.906163+0000","last_became_active":"2024-04-01T00:13:47.046387+0000","last_became_peered":"2024-04-01T00:13:47.046387+0000","last_unstale":"2024-04-01T00:13:48.906163+0000","last_undegraded":"2024-04-01T00:13:48.906163+0000","last_fullsized":"2024-04-01T00:13:48.906163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:44:37.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026803+0000","last_change":"2024-04-01T00:13:46.026873+0000","last_active":"2024-04-01T00:13:48.026803+0000","last_peered":"2024-04-01T00:13:48.026803+0000","last_clean":"2024-04-01T00:13:48.026803+0000","last_became_active":"2024-04-01T00:13:46.026564+0000","last_became_peered":"2024-04-01T00:13:46.026564+0000","last_unstale":"2024-04-01T00:13:48.026803+0000","last_undegraded":"2024-04-01T00:13:48.026803+0000","last_fullsized":"2024-04-01T00:13:48.026803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905168+0000","last_change":"2024-04-01T00:13:47.040797+0000","last_active":"2024-04-01T00:13:48.905168+0000","last_peered":"2024-04-01T00:13:48.905168+0000","last_clean":"2024-04-01T00:13:48.905168+0000","last_became_active":"2024-04-01T00:13:47.040224+0000","last_became_peered":"2024-04-01T00:13:47.040224+0000","last_unstale":"2024-04-01T00:13:48.905168+0000","last_undegraded":"2024-04-01T00:13:48.905168+0000","last_fullsized":"2024-04-01T00:13:48.905168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030476+0000","last_change":"2024-04-01T00:13:46.036254+0000","last_active":"2024-04-01T00:13:48.030476+0000","last_peered":"2024-04-01T00:13:48.030476+0000","last_clean":"2024-04-01T00:13:48.030476+0000","last_became_active":"2024-04-01T00:13:46.035969+0000","last_became_peered":"2024-04-01T00:13:46.035969+0000","last_unstale":"2024-04-01T00:13:48.030476+0000","last_undegraded":"2024-04-01T00:13:48.030476+0000","last_fullsized":"2024-04-01T00:13:48.030476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030469+0000","last_change":"2024-04-01T00:13:47.039818+0000","last_active":"2024-04-01T00:13:48.030469+0000","last_peered":"2024-04-01T00:13:48.030469+0000","last_clean":"2024-04-01T00:13:48.030469+0000","last_became_active":"2024-04-01T00:13:47.039157+0000","last_became_peered":"2024-04-01T00:13:47.039157+0000","last_unstale":"2024-04-01T00:13:48.030469+0000","last_undegraded":"2024-04-01T00:13:48.030469+0000","last_fullsized":"2024-04-01T00:13:48.030469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:01:22.390877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905588+0000","last_change":"2024-04-01T00:13:46.023660+0000","last_active":"2024-04-01T00:13:48.905588+0000","last_peered":"2024-04-01T00:13:48.905588+0000","last_clean":"2024-04-01T00:13:48.905588+0000","last_became_active":"2024-04-01T00:13:46.023368+0000","last_became_peered":"2024-04-01T00:13:46.023368+0000","last_unstale":"2024-04-01T00:13:48.905588+0000","last_undegraded":"2024-04-01T00:13:48.905588+0000","last_fullsized":"2024-04-01T00:13:48.905588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:11:34.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904995+0000","last_change":"2024-04-01T00:13:47.040468+0000","last_active":"2024-04-01T00:13:48.904995+0000","last_peered":"2024-04-01T00:13:48.904995+0000","last_clean":"2024-04-01T00:13:48.904995+0000","last_became_active":"2024-04-01T00:13:47.040198+0000","last_became_peered":"2024-04-01T00:13:47.040198+0000","last_unstale":"2024-04-01T00:13:48.904995+0000","last_undegraded":"2024-04-01T00:13:48.904995+0000","last_fullsized":"2024-04-01T00:13:48.904995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:39:21.192572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028251+0000","last_change":"2024-04-01T00:13:46.029828+0000","last_active":"2024-04-01T00:13:48.028251+0000","last_peered":"2024-04-01T00:13:48.028251+0000","last_clean":"2024-04-01T00:13:48.028251+0000","last_became_active":"2024-04-01T00:13:46.029514+0000","last_became_peered":"2024-04-01T00:13:46.029514+0000","last_unstale":"2024-04-01T00:13:48.028251+0000","last_undegraded":"2024-04-01T00:13:48.028251+0000","last_fullsized":"2024-04-01T00:13:48.028251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:16:56.432398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905327+0000","last_change":"2024-04-01T00:13:47.037521+0000","last_active":"2024-04-01T00:13:48.905327+0000","last_peered":"2024-04-01T00:13:48.905327+0000","last_clean":"2024-04-01T00:13:48.905327+0000","last_became_active":"2024-04-01T00:13:47.037208+0000","last_became_peered":"2024-04-01T00:13:47.037208+0000","last_unstale":"2024-04-01T00:13:48.905327+0000","last_undegraded":"2024-04-01T00:13:48.905327+0000","last_fullsized":"2024-04-01T00:13:48.905327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905466+0000","last_change":"2024-04-01T00:13:46.029659+0000","last_active":"2024-04-01T00:13:48.905466+0000","last_peered":"2024-04-01T00:13:48.905466+0000","last_clean":"2024-04-01T00:13:48.905466+0000","last_became_active":"2024-04-01T00:13:46.029376+0000","last_became_peered":"2024-04-01T00:13:46.029376+0000","last_unstale":"2024-04-01T00:13:48.905466+0000","last_undegraded":"2024-04-01T00:13:48.905466+0000","last_fullsized":"2024-04-01T00:13:48.905466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:37:01.341140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905503+0000","last_change":"2024-04-01T00:13:47.045515+0000","last_active":"2024-04-01T00:13:48.905503+0000","last_peered":"2024-04-01T00:13:48.905503+0000","last_clean":"2024-04-01T00:13:48.905503+0000","last_became_active":"2024-04-01T00:13:47.045138+0000","last_became_peered":"2024-04-01T00:13:47.045138+0000","last_unstale":"2024-04-01T00:13:48.905503+0000","last_undegraded":"2024-04-01T00:13:48.905503+0000","last_fullsized":"2024-04-01T00:13:48.905503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:32:58.179314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025530+0000","last_change":"2024-04-01T00:13:46.033447+0000","last_active":"2024-04-01T00:13:48.025530+0000","last_peered":"2024-04-01T00:13:48.025530+0000","last_clean":"2024-04-01T00:13:48.025530+0000","last_became_active":"2024-04-01T00:13:46.033126+0000","last_became_peered":"2024-04-01T00:13:46.033126+0000","last_unstale":"2024-04-01T00:13:48.025530+0000","last_undegraded":"2024-04-01T00:13:48.025530+0000","last_fullsized":"2024-04-01T00:13:48.025530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027072+0000","last_change":"2024-04-01T00:13:47.033643+0000","last_active":"2024-04-01T00:13:48.027072+0000","last_peered":"2024-04-01T00:13:48.027072+0000","last_clean":"2024-04-01T00:13:48.027072+0000","last_became_active":"2024-04-01T00:13:47.033350+0000","last_became_peered":"2024-04-01T00:13:47.033350+0000","last_unstale":"2024-04-01T00:13:48.027072+0000","last_undegraded":"2024-04-01T00:13:48.027072+0000","last_fullsized":"2024-04-01T00:13:48.027072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905480+0000","last_change":"2024-04-01T00:13:46.021048+0000","last_active":"2024-04-01T00:13:48.905480+0000","last_peered":"2024-04-01T00:13:48.905480+0000","last_clean":"2024-04-01T00:13:48.905480+0000","last_became_active":"2024-04-01T00:13:46.020612+0000","last_became_peered":"2024-04-01T00:13:46.020612+0000","last_unstale":"2024-04-01T00:13:48.905480+0000","last_undegraded":"2024-04-01T00:13:48.905480+0000","last_fullsized":"2024-04-01T00:13:48.905480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905112+0000","last_change":"2024-04-01T00:13:47.038059+0000","last_active":"2024-04-01T00:13:48.905112+0000","last_peered":"2024-04-01T00:13:48.905112+0000","last_clean":"2024-04-01T00:13:48.905112+0000","last_became_active":"2024-04-01T00:13:47.034213+0000","last_became_peered":"2024-04-01T00:13:47.034213+0000","last_unstale":"2024-04-01T00:13:48.905112+0000","last_undegraded":"2024-04-01T00:13:48.905112+0000","last_fullsized":"2024-04-01T00:13:48.905112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904882+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:13:48.904882+0000","last_peered":"2024-04-01T00:13:48.904882+0000","last_clean":"2024-04-01T00:13:48.904882+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:13:48.904882+0000","last_undegraded":"2024-04-01T00:13:48.904882+0000","last_fullsized":"2024-04-01T00:13:48.904882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026668+0000","last_change":"2024-04-01T00:13:46.019635+0000","last_active":"2024-04-01T00:13:48.026668+0000","last_peered":"2024-04-01T00:13:48.026668+0000","last_clean":"2024-04-01T00:13:48.026668+0000","last_became_active":"2024-04-01T00:13:46.019355+0000","last_became_peered":"2024-04-01T00:13:46.019355+0000","last_unstale":"2024-04-01T00:13:48.026668+0000","last_undegraded":"2024-04-01T00:13:48.026668+0000","last_fullsized":"2024-04-01T00:13:48.026668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:18.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906283+0000","last_change":"2024-04-01T00:13:47.034055+0000","last_active":"2024-04-01T00:13:48.906283+0000","last_peered":"2024-04-01T00:13:48.906283+0000","last_clean":"2024-04-01T00:13:48.906283+0000","last_became_active":"2024-04-01T00:13:47.033778+0000","last_became_peered":"2024-04-01T00:13:47.033778+0000","last_unstale":"2024-04-01T00:13:48.906283+0000","last_undegraded":"2024-04-01T00:13:48.906283+0000","last_fullsized":"2024-04-01T00:13:48.906283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904818+0000","last_change":"2024-04-01T00:13:46.037663+0000","last_active":"2024-04-01T00:13:48.904818+0000","last_peered":"2024-04-01T00:13:48.904818+0000","last_clean":"2024-04-01T00:13:48.904818+0000","last_became_active":"2024-04-01T00:13:46.037388+0000","last_became_peered":"2024-04-01T00:13:46.037388+0000","last_unstale":"2024-04-01T00:13:48.904818+0000","last_undegraded":"2024-04-01T00:13:48.904818+0000","last_fullsized":"2024-04-01T00:13:48.904818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906330+0000","last_change":"2024-04-01T00:13:47.038068+0000","last_active":"2024-04-01T00:13:48.906330+0000","last_peered":"2024-04-01T00:13:48.906330+0000","last_clean":"2024-04-01T00:13:48.906330+0000","last_became_active":"2024-04-01T00:13:47.037774+0000","last_became_peered":"2024-04-01T00:13:47.037774+0000","last_unstale":"2024-04-01T00:13:48.906330+0000","last_undegraded":"2024-04-01T00:13:48.906330+0000","last_fullsized":"2024-04-01T00:13:48.906330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:17:27.372297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905353+0000","last_change":"2024-04-01T00:13:46.029621+0000","last_active":"2024-04-01T00:13:48.905353+0000","last_peered":"2024-04-01T00:13:48.905353+0000","last_clean":"2024-04-01T00:13:48.905353+0000","last_became_active":"2024-04-01T00:13:46.029319+0000","last_became_peered":"2024-04-01T00:13:46.029319+0000","last_unstale":"2024-04-01T00:13:48.905353+0000","last_undegraded":"2024-04-01T00:13:48.905353+0000","last_fullsized":"2024-04-01T00:13:48.905353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:36:55.173387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025440+0000","last_change":"2024-04-01T00:13:47.030092+0000","last_active":"2024-04-01T00:13:48.025440+0000","last_peered":"2024-04-01T00:13:48.025440+0000","last_clean":"2024-04-01T00:13:48.025440+0000","last_became_active":"2024-04-01T00:13:47.029837+0000","last_became_peered":"2024-04-01T00:13:47.029837+0000","last_unstale":"2024-04-01T00:13:48.025440+0000","last_undegraded":"2024-04-01T00:13:48.025440+0000","last_fullsized":"2024-04-01T00:13:48.025440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:41:57.634359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026885+0000","last_change":"2024-04-01T00:13:46.027134+0000","last_active":"2024-04-01T00:13:48.026885+0000","last_peered":"2024-04-01T00:13:48.026885+0000","last_clean":"2024-04-01T00:13:48.026885+0000","last_became_active":"2024-04-01T00:13:46.026855+0000","last_became_peered":"2024-04-01T00:13:46.026855+0000","last_unstale":"2024-04-01T00:13:48.026885+0000","last_undegraded":"2024-04-01T00:13:48.026885+0000","last_fullsized":"2024-04-01T00:13:48.026885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028071+0000","last_change":"2024-04-01T00:13:47.041868+0000","last_active":"2024-04-01T00:13:48.028071+0000","last_peered":"2024-04-01T00:13:48.028071+0000","last_clean":"2024-04-01T00:13:48.028071+0000","last_became_active":"2024-04-01T00:13:47.040849+0000","last_became_peered":"2024-04-01T00:13:47.040849+0000","last_unstale":"2024-04-01T00:13:48.028071+0000","last_undegraded":"2024-04-01T00:13:48.028071+0000","last_fullsized":"2024-04-01T00:13:48.028071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:54.759711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025337+0000","last_change":"2024-04-01T00:13:46.026672+0000","last_active":"2024-04-01T00:13:48.025337+0000","last_peered":"2024-04-01T00:13:48.025337+0000","last_clean":"2024-04-01T00:13:48.025337+0000","last_became_active":"2024-04-01T00:13:46.026365+0000","last_became_peered":"2024-04-01T00:13:46.026365+0000","last_unstale":"2024-04-01T00:13:48.025337+0000","last_undegraded":"2024-04-01T00:13:48.025337+0000","last_fullsized":"2024-04-01T00:13:48.025337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027240+0000","last_change":"2024-04-01T00:13:47.028664+0000","last_active":"2024-04-01T00:13:48.027240+0000","last_peered":"2024-04-01T00:13:48.027240+0000","last_clean":"2024-04-01T00:13:48.027240+0000","last_became_active":"2024-04-01T00:13:47.028387+0000","last_became_peered":"2024-04-01T00:13:47.028387+0000","last_unstale":"2024-04-01T00:13:48.027240+0000","last_undegraded":"2024-04-01T00:13:48.027240+0000","last_fullsized":"2024-04-01T00:13:48.027240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:24:33.181243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905298+0000","last_change":"2024-04-01T00:13:46.037508+0000","last_active":"2024-04-01T00:13:48.905298+0000","last_peered":"2024-04-01T00:13:48.905298+0000","last_clean":"2024-04-01T00:13:48.905298+0000","last_became_active":"2024-04-01T00:13:46.037083+0000","last_became_peered":"2024-04-01T00:13:46.037083+0000","last_unstale":"2024-04-01T00:13:48.905298+0000","last_undegraded":"2024-04-01T00:13:48.905298+0000","last_fullsized":"2024-04-01T00:13:48.905298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:42.737710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025575+0000","last_change":"2024-04-01T00:13:47.027434+0000","last_active":"2024-04-01T00:13:48.025575+0000","last_peered":"2024-04-01T00:13:48.025575+0000","last_clean":"2024-04-01T00:13:48.025575+0000","last_became_active":"2024-04-01T00:13:47.027134+0000","last_became_peered":"2024-04-01T00:13:47.027134+0000","last_unstale":"2024-04-01T00:13:48.025575+0000","last_undegraded":"2024-04-01T00:13:48.025575+0000","last_fullsized":"2024-04-01T00:13:48.025575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:18:49.350147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030010+0000","last_change":"2024-04-01T00:13:46.029352+0000","last_active":"2024-04-01T00:13:48.030010+0000","last_peered":"2024-04-01T00:13:48.030010+0000","last_clean":"2024-04-01T00:13:48.030010+0000","last_became_active":"2024-04-01T00:13:46.029053+0000","last_became_peered":"2024-04-01T00:13:46.029053+0000","last_unstale":"2024-04-01T00:13:48.030010+0000","last_undegraded":"2024-04-01T00:13:48.030010+0000","last_fullsized":"2024-04-01T00:13:48.030010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906349+0000","last_change":"2024-04-01T00:13:47.045641+0000","last_active":"2024-04-01T00:13:48.906349+0000","last_peered":"2024-04-01T00:13:48.906349+0000","last_clean":"2024-04-01T00:13:48.906349+0000","last_became_active":"2024-04-01T00:13:47.045348+0000","last_became_peered":"2024-04-01T00:13:47.045348+0000","last_unstale":"2024-04-01T00:13:48.906349+0000","last_undegraded":"2024-04-01T00:13:48.906349+0000","last_fullsized":"2024-04-01T00:13:48.906349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:22:59.667325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030805+0000","last_change":"2024-04-01T00:13:46.037865+0000","last_active":"2024-04-01T00:13:48.030805+0000","last_peered":"2024-04-01T00:13:48.030805+0000","last_clean":"2024-04-01T00:13:48.030805+0000","last_became_active":"2024-04-01T00:13:46.037564+0000","last_became_peered":"2024-04-01T00:13:46.037564+0000","last_unstale":"2024-04-01T00:13:48.030805+0000","last_undegraded":"2024-04-01T00:13:48.030805+0000","last_fullsized":"2024-04-01T00:13:48.030805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906532+0000","last_change":"2024-04-01T00:13:47.038745+0000","last_active":"2024-04-01T00:13:48.906532+0000","last_peered":"2024-04-01T00:13:48.906532+0000","last_clean":"2024-04-01T00:13:48.906532+0000","last_became_active":"2024-04-01T00:13:47.038440+0000","last_became_peered":"2024-04-01T00:13:47.038440+0000","last_unstale":"2024-04-01T00:13:48.906532+0000","last_undegraded":"2024-04-01T00:13:48.906532+0000","last_fullsized":"2024-04-01T00:13:48.906532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.904810+0000","last_change":"2024-04-01T00:13:46.037408+0000","last_active":"2024-04-01T00:13:48.904810+0000","last_peered":"2024-04-01T00:13:48.904810+0000","last_clean":"2024-04-01T00:13:48.904810+0000","last_became_active":"2024-04-01T00:13:46.037120+0000","last_became_peered":"2024-04-01T00:13:46.037120+0000","last_unstale":"2024-04-01T00:13:48.904810+0000","last_undegraded":"2024-04-01T00:13:48.904810+0000","last_fullsized":"2024-04-01T00:13:48.904810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:49:24.801239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.028332+0000","last_change":"2024-04-01T00:13:47.032590+0000","last_active":"2024-04-01T00:13:48.028332+0000","last_peered":"2024-04-01T00:13:48.028332+0000","last_clean":"2024-04-01T00:13:48.028332+0000","last_became_active":"2024-04-01T00:13:47.032294+0000","last_became_peered":"2024-04-01T00:13:47.032294+0000","last_unstale":"2024-04-01T00:13:48.028332+0000","last_undegraded":"2024-04-01T00:13:48.028332+0000","last_fullsized":"2024-04-01T00:13:48.028332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:02:00.483638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025536+0000","last_change":"2024-04-01T00:13:46.036386+0000","last_active":"2024-04-01T00:13:48.025536+0000","last_peered":"2024-04-01T00:13:48.025536+0000","last_clean":"2024-04-01T00:13:48.025536+0000","last_became_active":"2024-04-01T00:13:46.036055+0000","last_became_peered":"2024-04-01T00:13:46.036055+0000","last_unstale":"2024-04-01T00:13:48.025536+0000","last_undegraded":"2024-04-01T00:13:48.025536+0000","last_fullsized":"2024-04-01T00:13:48.025536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:01:59.478277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905628+0000","last_change":"2024-04-01T00:13:47.027936+0000","last_active":"2024-04-01T00:13:48.905628+0000","last_peered":"2024-04-01T00:13:48.905628+0000","last_clean":"2024-04-01T00:13:48.905628+0000","last_became_active":"2024-04-01T00:13:47.027628+0000","last_became_peered":"2024-04-01T00:13:47.027628+0000","last_unstale":"2024-04-01T00:13:48.905628+0000","last_undegraded":"2024-04-01T00:13:48.905628+0000","last_fullsized":"2024-04-01T00:13:48.905628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:17:24.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.025431+0000","last_change":"2024-04-01T00:13:46.028586+0000","last_active":"2024-04-01T00:13:48.025431+0000","last_peered":"2024-04-01T00:13:48.025431+0000","last_clean":"2024-04-01T00:13:48.025431+0000","last_became_active":"2024-04-01T00:13:46.028309+0000","last_became_peered":"2024-04-01T00:13:46.028309+0000","last_unstale":"2024-04-01T00:13:48.025431+0000","last_undegraded":"2024-04-01T00:13:48.025431+0000","last_fullsized":"2024-04-01T00:13:48.025431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:07:12.508437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906125+0000","last_change":"2024-04-01T00:13:47.042379+0000","last_active":"2024-04-01T00:13:48.906125+0000","last_peered":"2024-04-01T00:13:48.906125+0000","last_clean":"2024-04-01T00:13:48.906125+0000","last_became_active":"2024-04-01T00:13:47.042018+0000","last_became_peered":"2024-04-01T00:13:47.042018+0000","last_unstale":"2024-04-01T00:13:48.906125+0000","last_undegraded":"2024-04-01T00:13:48.906125+0000","last_fullsized":"2024-04-01T00:13:48.906125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:04:18.081336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905276+0000","last_change":"2024-04-01T00:13:46.037179+0000","last_active":"2024-04-01T00:13:48.905276+0000","last_peered":"2024-04-01T00:13:48.905276+0000","last_clean":"2024-04-01T00:13:48.905276+0000","last_became_active":"2024-04-01T00:13:46.036168+0000","last_became_peered":"2024-04-01T00:13:46.036168+0000","last_unstale":"2024-04-01T00:13:48.905276+0000","last_undegraded":"2024-04-01T00:13:48.905276+0000","last_fullsized":"2024-04-01T00:13:48.905276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:56.460345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.030533+0000","last_change":"2024-04-01T00:13:47.031588+0000","last_active":"2024-04-01T00:13:48.030533+0000","last_peered":"2024-04-01T00:13:48.030533+0000","last_clean":"2024-04-01T00:13:48.030533+0000","last_became_active":"2024-04-01T00:13:47.031297+0000","last_became_peered":"2024-04-01T00:13:47.031297+0000","last_unstale":"2024-04-01T00:13:48.030533+0000","last_undegraded":"2024-04-01T00:13:48.030533+0000","last_fullsized":"2024-04-01T00:13:48.030533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:06:24.419804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.027184+0000","last_change":"2024-04-01T00:13:47.036405+0000","last_active":"2024-04-01T00:13:48.027184+0000","last_peered":"2024-04-01T00:13:48.027184+0000","last_clean":"2024-04-01T00:13:48.027184+0000","last_became_active":"2024-04-01T00:13:47.036144+0000","last_became_peered":"2024-04-01T00:13:47.036144+0000","last_unstale":"2024-04-01T00:13:48.027184+0000","last_undegraded":"2024-04-01T00:13:48.027184+0000","last_fullsized":"2024-04-01T00:13:48.027184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:38:38.572066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905062+0000","last_change":"2024-04-01T00:13:46.036339+0000","last_active":"2024-04-01T00:13:48.905062+0000","last_peered":"2024-04-01T00:13:48.905062+0000","last_clean":"2024-04-01T00:13:48.905062+0000","last_became_active":"2024-04-01T00:13:46.036038+0000","last_became_peered":"2024-04-01T00:13:46.036038+0000","last_unstale":"2024-04-01T00:13:48.905062+0000","last_undegraded":"2024-04-01T00:13:48.905062+0000","last_fullsized":"2024-04-01T00:13:48.905062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:43:50.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.026918+0000","last_change":"2024-04-01T00:13:47.037119+0000","last_active":"2024-04-01T00:13:48.026918+0000","last_peered":"2024-04-01T00:13:48.026918+0000","last_clean":"2024-04-01T00:13:48.026918+0000","last_became_active":"2024-04-01T00:13:47.036860+0000","last_became_peered":"2024-04-01T00:13:47.036860+0000","last_unstale":"2024-04-01T00:13:48.026918+0000","last_undegraded":"2024-04-01T00:13:48.026918+0000","last_fullsized":"2024-04-01T00:13:48.026918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:49:54.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905203+0000","last_change":"2024-04-01T00:13:46.036988+0000","last_active":"2024-04-01T00:13:48.905203+0000","last_peered":"2024-04-01T00:13:48.905203+0000","last_clean":"2024-04-01T00:13:48.905203+0000","last_became_active":"2024-04-01T00:13:46.036696+0000","last_became_peered":"2024-04-01T00:13:46.036696+0000","last_unstale":"2024-04-01T00:13:48.905203+0000","last_undegraded":"2024-04-01T00:13:48.905203+0000","last_fullsized":"2024-04-01T00:13:48.905203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:35:19.282337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906528+0000","last_change":"2024-04-01T00:13:47.041150+0000","last_active":"2024-04-01T00:13:48.906528+0000","last_peered":"2024-04-01T00:13:48.906528+0000","last_clean":"2024-04-01T00:13:48.906528+0000","last_became_active":"2024-04-01T00:13:47.040826+0000","last_became_peered":"2024-04-01T00:13:47.040826+0000","last_unstale":"2024-04-01T00:13:48.906528+0000","last_undegraded":"2024-04-01T00:13:48.906528+0000","last_fullsized":"2024-04-01T00:13:48.906528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:04:32.284194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906154+0000","last_change":"2024-04-01T00:13:46.032149+0000","last_active":"2024-04-01T00:13:48.906154+0000","last_peered":"2024-04-01T00:13:48.906154+0000","last_clean":"2024-04-01T00:13:48.906154+0000","last_became_active":"2024-04-01T00:13:46.031821+0000","last_became_peered":"2024-04-01T00:13:46.031821+0000","last_unstale":"2024-04-01T00:13:48.906154+0000","last_undegraded":"2024-04-01T00:13:48.906154+0000","last_fullsized":"2024-04-01T00:13:48.906154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:53.754350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.905922+0000","last_change":"2024-04-01T00:13:47.031502+0000","last_active":"2024-04-01T00:13:48.905922+0000","last_peered":"2024-04-01T00:13:48.905922+0000","last_clean":"2024-04-01T00:13:48.905922+0000","last_became_active":"2024-04-01T00:13:47.030468+0000","last_became_peered":"2024-04-01T00:13:47.030468+0000","last_unstale":"2024-04-01T00:13:48.905922+0000","last_undegraded":"2024-04-01T00:13:48.905922+0000","last_fullsized":"2024-04-01T00:13:48.905922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:46.007027+0000","last_clean_scrub_stamp":"2024-04-01T00:13:46.007027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:03:34.589236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-01T00:13:48.906670+0000","last_change":"2024-04-01T00:13:46.038424+0000","last_active":"2024-04-01T00:13:48.906670+0000","last_peered":"2024-04-01T00:13:48.906670+0000","last_clean":"2024-04-01T00:13:48.906670+0000","last_became_active":"2024-04-01T00:13:46.038079+0000","last_became_peered":"2024-04-01T00:13:46.038079+0000","last_unstale":"2024-04-01T00:13:48.906670+0000","last_undegraded":"2024-04-01T00:13:48.906670+0000","last_fullsized":"2024-04-01T00:13:48.906670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:45.001667+0000","last_clean_scrub_stamp":"2024-04-01T00:13:45.001667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:44:48.269461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:13:57.821 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-01T00:13:57.821 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-01T00:13:57.821 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-01T00:13:57.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-01T00:13:57.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.958+0000 7f2829e40640 1 Processor -- start 2024-04-01T00:13:57.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.958+0000 7f2829e40640 1 -- start start 2024-04-01T00:13:57.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2829e40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 0x7f282406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2829e40640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28240b5d20 con 0x7f28240b4350 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2829e40640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2824161e10 con 0x7f2824147420 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2829e40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f28241620b0 con 0x7f282406b7d0 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28237fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f28240b4350 0x7f282406b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48056/0 (socket says 172.21.15.64:48056) 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28237fe640 1 -- 172.21.15.64:0/3633819340 learned_addr learned my addr 172.21.15.64:0/3633819340 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2822ffd640 1 --2- 172.21.15.64:0/3633819340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 0x7f282406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253545923 0 0) 0x7f28240b5d20 con 0x7f28240b4350 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f280c003520 con 0x7f28240b4350 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944554126 0 0) 0x7f2824161e10 con 0x7f2824147420 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28240b5d20 con 0x7f2824147420 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2822ffd640 1 -- 172.21.15.64:0/3633819340 >> v1:172.21.15.100:6789/0 conn(0x7f2824147420 legacy=0x7f282406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2822ffd640 1 -- 172.21.15.64:0/3633819340 >> v1:172.21.15.64:6789/0 conn(0x7f28240b4350 legacy=0x7f282406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.959+0000 7f2822ffd640 1 -- 172.21.15.64:0/3633819340 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2824162390 con 0x7f282406b7d0 2024-04-01T00:13:57.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.960+0000 7f2822ffd640 1 --2- 172.21.15.64:0/3633819340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 0x7f282406de60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28180097d0 tx=0x7f281802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d3964975146928 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.960+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818035020 con 0x7f282406b7d0 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.960+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2818031890 con 0x7f282406b7d0 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.960+0000 7f28227fc640 1 -- 172.21.15.64:0/3633819340 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818031c60 con 0x7f282406b7d0 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 -- 172.21.15.64:0/3633819340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 msgr2=0x7f282406de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 --2- 172.21.15.64:0/3633819340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 0x7f282406de60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28180097d0 tx=0x7f281802b930 comp rx=0 tx=0).stop 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 -- 172.21.15.64:0/3633819340 shutdown_connections 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 --2- 172.21.15.64:0/3633819340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f282406b7d0 0x7f282406de60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 -- 172.21.15.64:0/3633819340 >> 172.21.15.64:0/3633819340 conn(0x7f28240b2760 msgr2=0x7f28240b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 -- 172.21.15.64:0/3633819340 shutdown_connections 2024-04-01T00:13:57.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.961+0000 7f2829e40640 1 -- 172.21.15.64:0/3633819340 wait complete. 2024-04-01T00:13:57.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.962+0000 7f2829e40640 1 Processor -- start 2024-04-01T00:13:57.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.962+0000 7f2829e40640 1 -- start start 2024-04-01T00:13:57.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28240b4350 0x7f282413bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2824147420 0x7f28240ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2824147970 con 0x7f282406b7d0 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2824162950 con 0x7f2824147420 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2829e40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2824147af0 con 0x7f28240b4350 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51362/0 (socket says 172.21.15.64:51362) 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 -- 172.21.15.64:0/4005407388 learned_addr learned my addr 172.21.15.64:0/4005407388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2822ffd640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28240b4350 0x7f282413bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f2823fff640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2824147420 0x7f28240ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 -- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28240b4350 msgr2=0x7f282413bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28240b4350 0x7f282413bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 -- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2824147420 msgr2=0x7f28240ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2824147420 0x7f28240ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 -- 172.21.15.64:0/4005407388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2824140600 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f28237fe640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f281400c530 tx=0x7f281400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.963+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814017070 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.964+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2814005ce0 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.964+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28241408e0 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.964+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814007270 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.964+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2824140ea0 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.965+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28240681a0 con 0x7f282406b7d0 2024-04-01T00:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.965+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f281401a070 con 0x7f282406b7d0 2024-04-01T00:13:57.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.965+0000 7f280bfff640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 0x7f281006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:57.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.965+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2814089b30 con 0x7f282406b7d0 2024-04-01T00:13:57.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.966+0000 7f2822ffd640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 0x7f281006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:57.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.966+0000 7f2822ffd640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 0x7f281006de50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2818036000 tx=0x7f2818008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:57.970+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f28140580f0 con 0x7f282406b7d0 2024-04-01T00:13:58.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.187+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f282406fbc0 con 0x7f282406b7d0 2024-04-01T00:13:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.188+0000 7f280bfff640 1 -- 172.21.15.64:0/4005407388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f281405c190 con 0x7f282406b7d0 2024-04-01T00:13:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:58.189 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 msgr2=0x7f281006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 0x7f281006de50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2818036000 tx=0x7f2818008030 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 msgr2=0x7f2824160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f281400c530 tx=0x7f281400c9f0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 shutdown_connections 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f281006b990 0x7f281006de50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2824147420 0x7f28240ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28240b4350 0x7f282413bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 --2- 172.21.15.64:0/4005407388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f282406b7d0 0x7f2824160950 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 >> 172.21.15.64:0/4005407388 conn(0x7f28240b2760 msgr2=0x7f28240b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 shutdown_connections 2024-04-01T00:13:58.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.190+0000 7f2829e40640 1 -- 172.21.15.64:0/4005407388 wait complete. 2024-04-01T00:13:58.201 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-01T00:13:58.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.302+0000 7f8499c15640 1 Processor -- start 2024-04-01T00:13:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 -- start start 2024-04-01T00:13:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84941619e0 0x7f84940d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84940cfae0 0x7f84941620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84940b3cb0 con 0x7f84940b3820 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f84940d2ba0 con 0x7f84940cfae0 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8499c15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f84940d2e80 con 0x7f84941619e0 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8493fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84940cfae0 0x7f84941620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8493fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84940cfae0 0x7f84941620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51370/0 (socket says 172.21.15.64:51370) 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f84937fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84941619e0 0x7f84940d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.303+0000 7f8493fff640 1 -- 172.21.15.64:0/3983670026 learned_addr learned my addr 172.21.15.64:0/3983670026 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.304+0000 7f8493fff640 1 -- 172.21.15.64:0/3983670026 >> v2:172.21.15.100:3300/0 conn(0x7f84941619e0 msgr2=0x7f84940d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.304+0000 7f8493fff640 1 --2- 172.21.15.64:0/3983670026 >> v2:172.21.15.100:3300/0 conn(0x7f84941619e0 0x7f84940d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.304+0000 7f8493fff640 1 -- 172.21.15.64:0/3983670026 >> v1:172.21.15.100:6789/0 conn(0x7f84940b3820 legacy=0x7f84940cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.304+0000 7f8493fff640 1 -- 172.21.15.64:0/3983670026 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84940ceeb0 con 0x7f84940cfae0 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.304+0000 7f8493fff640 1 --2- 172.21.15.64:0/3983670026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940cfae0 0x7f84941620a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84940b6b40 tx=0x7f848c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77893eb9c8efd8c server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f84927fc640 1 -- 172.21.15.64:0/3983670026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c02ca80 con 0x7f84940cfae0 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f84927fc640 1 -- 172.21.15.64:0/3983670026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c02cc20 con 0x7f84940cfae0 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f84927fc640 1 -- 172.21.15.64:0/3983670026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c0343f0 con 0x7f84940cfae0 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 -- 172.21.15.64:0/3983670026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940cfae0 msgr2=0x7f84941620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 --2- 172.21.15.64:0/3983670026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940cfae0 0x7f84941620a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84940b6b40 tx=0x7f848c02bee0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 -- 172.21.15.64:0/3983670026 shutdown_connections 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 --2- 172.21.15.64:0/3983670026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940cfae0 0x7f84941620a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 --2- 172.21.15.64:0/3983670026 >> v2:172.21.15.100:3300/0 conn(0x7f84941619e0 0x7f84940d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 -- 172.21.15.64:0/3983670026 >> 172.21.15.64:0/3983670026 conn(0x7f849409d290 msgr2=0x7f84940d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 -- 172.21.15.64:0/3983670026 shutdown_connections 2024-04-01T00:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.305+0000 7f8499c15640 1 -- 172.21.15.64:0/3983670026 wait complete. 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.306+0000 7f8499c15640 1 Processor -- start 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 -- start start 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84940cfae0 0x7f84941f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84941619e0 0x7f84941502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84940b31b0 con 0x7f84940b3820 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84940d2890 con 0x7f84940cfae0 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8499c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f84940ceb10 con 0x7f84941619e0 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f84937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f84937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51376/0 (socket says 172.21.15.64:51376) 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f84937fe640 1 -- 172.21.15.64:0/435440650 learned_addr learned my addr 172.21.15.64:0/435440650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.307+0000 7f8492ffd640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84940cfae0 0x7f84941f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f8493fff640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84941619e0 0x7f84941502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 -- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84941619e0 msgr2=0x7f84941502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84941619e0 0x7f84941502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 -- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84940cfae0 msgr2=0x7f84941f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84940cfae0 0x7f84941f81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 -- 172.21.15.64:0/435440650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84940cdcc0 con 0x7f84940b3820 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f84937fe640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f848400e870 tx=0x7f848400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484017070 con 0x7f84940b3820 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848400c940 con 0x7f84940b3820 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848400cd30 con 0x7f84940b3820 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8494150c10 con 0x7f84940b3820 2024-04-01T00:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.308+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941f79a0 con 0x7f84940b3820 2024-04-01T00:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.309+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84940681a0 con 0x7f84940b3820 2024-04-01T00:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.310+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f848401a070 con 0x7f84940b3820 2024-04-01T00:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.310+0000 7f848bfff640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 0x7f847406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.310+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f848408a590 con 0x7f84940b3820 2024-04-01T00:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.310+0000 7f8492ffd640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 0x7f847406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.311+0000 7f8492ffd640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 0x7f847406de80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f849415d280 tx=0x7f847c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.315+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8484058be0 con 0x7f84940b3820 2024-04-01T00:13:58.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.495+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f849409d730 con 0x7f84940b3820 2024-04-01T00:13:58.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.496+0000 7f848bfff640 1 -- 172.21.15.64:0/435440650 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f848405cc80 con 0x7f84940b3820 2024-04-01T00:13:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:58.498 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:13:58.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 msgr2=0x7f847406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 0x7f847406de80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f849415d280 tx=0x7f847c009040 comp rx=0 tx=0).stop 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 msgr2=0x7f84940cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f848400e870 tx=0x7f848400ed30 comp rx=0 tx=0).stop 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 shutdown_connections 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f847406b9c0 0x7f847406de80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84941619e0 0x7f84941502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84940cfae0 0x7f84941f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 --2- 172.21.15.64:0/435440650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84940b3820 0x7f84940cb760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.498+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 >> 172.21.15.64:0/435440650 conn(0x7f849409d290 msgr2=0x7f84940b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.499+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 shutdown_connections 2024-04-01T00:13:58.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.499+0000 7f8499c15640 1 -- 172.21.15.64:0/435440650 wait complete. 2024-04-01T00:13:58.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:13:58.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:13:58.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 Processor -- start 2024-04-01T00:13:58.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 -- start start 2024-04-01T00:13:58.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 0x7f481c168f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481c0b52c0 con 0x7f481c161a50 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481c0b55a0 con 0x7f481c0b2df0 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f48221d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f481c0b5070 con 0x7f481c0d0f20 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f481b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f481c161a50 0x7f481c0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48058/0 (socket says 172.21.15.64:48058) 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.617+0000 7f481b7fe640 1 -- 172.21.15.64:0/3859968885 learned_addr learned my addr 172.21.15.64:0/3859968885 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481bfff640 1 --2- 172.21.15.64:0/3859968885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 0x7f481c168f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2135136142 0 0) 0x7f481c0b52c0 con 0x7f481c161a50 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4804003520 con 0x7f481c161a50 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481bfff640 1 -- 172.21.15.64:0/3859968885 >> v1:172.21.15.100:6789/0 conn(0x7f481c0b2df0 legacy=0x7f481c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481bfff640 1 -- 172.21.15.64:0/3859968885 >> v1:172.21.15.64:6789/0 conn(0x7f481c161a50 legacy=0x7f481c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907677910 0 0) 0x7f481c0b55a0 con 0x7f481c0b2df0 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.618+0000 7f481bfff640 1 -- 172.21.15.64:0/3859968885 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c0d45a0 con 0x7f481c0d0f20 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f481bfff640 1 --2- 172.21.15.64:0/3859968885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 0x7f481c168f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4814009dc0 tx=0x7f481402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38c00d28ae6b9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814039070 con 0x7f481c0d0f20 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481402cc40 con 0x7f481c0d0f20 2024-04-01T00:13:58.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f481a7fc640 1 -- 172.21.15.64:0/3859968885 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814034440 con 0x7f481c0d0f20 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 -- 172.21.15.64:0/3859968885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 msgr2=0x7f481c168f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 --2- 172.21.15.64:0/3859968885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 0x7f481c168f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4814009dc0 tx=0x7f481402be90 comp rx=0 tx=0).stop 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 -- 172.21.15.64:0/3859968885 shutdown_connections 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 --2- 172.21.15.64:0/3859968885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0d0f20 0x7f481c168f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 -- 172.21.15.64:0/3859968885 >> 172.21.15.64:0/3859968885 conn(0x7f481c09d1c0 msgr2=0x7f481c0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 -- 172.21.15.64:0/3859968885 shutdown_connections 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.619+0000 7f48221d0640 1 -- 172.21.15.64:0/3859968885 wait complete. 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.620+0000 7f48221d0640 1 Processor -- start 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 -- start start 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0b2df0 0x7f481c1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f481c161a50 0x7f481c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f481c0b4950 con 0x7f481c0d0f20 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f481c0d4bf0 con 0x7f481c161a50 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f48221d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f481c0b4ad0 con 0x7f481c0b2df0 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f481affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f481affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51378/0 (socket says 172.21.15.64:51378) 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.621+0000 7f481affd640 1 -- 172.21.15.64:0/1602907987 learned_addr learned my addr 172.21.15.64:0/1602907987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481b7fe640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0b2df0 0x7f481c1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481bfff640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f481c161a50 0x7f481c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 -- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0b2df0 msgr2=0x7f481c1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0b2df0 0x7f481c1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 -- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f481c161a50 msgr2=0x7f481c152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f481c161a50 0x7f481c152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 -- 172.21.15.64:0/1602907987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c1f7a00 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f481affd640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f481000c410 tx=0x7f481000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810017070 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100035f0 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c1f7ce0 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100039e0 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.622+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c0b9a30 con 0x7f481c0d0f20 2024-04-01T00:13:58.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.623+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c0681a0 con 0x7f481c0d0f20 2024-04-01T00:13:58.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.623+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f481001a070 con 0x7f481c0d0f20 2024-04-01T00:13:58.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.624+0000 7f4803fff640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 0x7f480806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.624+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4810089b90 con 0x7f481c0d0f20 2024-04-01T00:13:58.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.624+0000 7f481b7fe640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 0x7f480806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.624+0000 7f481b7fe640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 0x7f480806de50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f480c005880 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.628+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f48100581e0 con 0x7f481c0d0f20 2024-04-01T00:13:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.808+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f481c1f3d90 con 0x7f481c0d0f20 2024-04-01T00:13:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.808+0000 7f4803fff640 1 -- 172.21.15.64:0/1602907987 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f481005c280 con 0x7f481c0d0f20 2024-04-01T00:13:58.809 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:13:58.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 msgr2=0x7f480806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 0x7f480806de50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f480c005880 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-04-01T00:13:58.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 msgr2=0x7f481c1f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f481000c410 tx=0x7f481000c8d0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 shutdown_connections 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f480806b990 0x7f480806de50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f481c161a50 0x7f481c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f481c0d0f20 0x7f481c1f6f60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 --2- 172.21.15.64:0/1602907987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481c0b2df0 0x7f481c1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 >> 172.21.15.64:0/1602907987 conn(0x7f481c09d1c0 msgr2=0x7f481c160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 shutdown_connections 2024-04-01T00:13:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.810+0000 7f48221d0640 1 -- 172.21.15.64:0/1602907987 wait complete. 2024-04-01T00:13:58.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:13:58.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.926+0000 7f52fff8a640 1 Processor -- start 2024-04-01T00:13:58.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.926+0000 7f52fff8a640 1 -- start start 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fff8a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fff8a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52f8161ad0 0x7f52f80d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fff8a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f80d18f0 con 0x7f52f80b4fb0 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fff8a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f52f80b4300 con 0x7f52f8161ad0 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fff8a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f52f80b45e0 con 0x7f52f81616f0 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52f8161ad0 0x7f52f80d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52f8161ad0 0x7f52f80d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51390/0 (socket says 172.21.15.64:51390) 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 -- 172.21.15.64:0/417087733 learned_addr learned my addr 172.21.15.64:0/417087733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fd525640 1 --1- 172.21.15.64:0/417087733 >> v1:172.21.15.64:6789/0 conn(0x7f52f80b4fb0 0x7f52f80d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48060/0 (socket says 172.21.15.64:48060) 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fdd26640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 -- 172.21.15.64:0/417087733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 msgr2=0x7f52f80d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 -- 172.21.15.64:0/417087733 >> v1:172.21.15.64:6789/0 conn(0x7f52f80b4fb0 legacy=0x7f52f80d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fe527640 1 -- 172.21.15.64:0/417087733 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f80b48c0 con 0x7f52f8161ad0 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fcd24640 1 -- 172.21.15.64:0/417087733 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138973885 0 0) 0x7f52f80d18f0 con 0x7f52f80b4fb0 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.927+0000 7f52fdd26640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.928+0000 7f52fe527640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f8161ad0 0x7f52f80d1040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009e50 tx=0x7f52e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ba8663e8b37be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.928+0000 7f52fcd24640 1 -- 172.21.15.64:0/417087733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e802ca70 con 0x7f52f8161ad0 2024-04-01T00:13:58.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.928+0000 7f52fcd24640 1 -- 172.21.15.64:0/417087733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e802cc10 con 0x7f52f8161ad0 2024-04-01T00:13:58.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.928+0000 7f52fcd24640 1 -- 172.21.15.64:0/417087733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e80343f0 con 0x7f52f8161ad0 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 -- 172.21.15.64:0/417087733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f8161ad0 msgr2=0x7f52f80d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f8161ad0 0x7f52f80d1040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009e50 tx=0x7f52e802bed0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 -- 172.21.15.64:0/417087733 shutdown_connections 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f8161ad0 0x7f52f80d1040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 --2- 172.21.15.64:0/417087733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 -- 172.21.15.64:0/417087733 >> 172.21.15.64:0/417087733 conn(0x7f52f80d3f60 msgr2=0x7f52f80d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 -- 172.21.15.64:0/417087733 shutdown_connections 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.929+0000 7f52fff8a640 1 -- 172.21.15.64:0/417087733 wait complete. 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.930+0000 7f52fff8a640 1 Processor -- start 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.930+0000 7f52fff8a640 1 -- start start 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f8161ad0 0x7f52f81f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80d1d70 con 0x7f52f80b4fb0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80d1580 con 0x7f52f8161ad0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f52f80d1ef0 con 0x7f52f81616f0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51404/0 (socket says 172.21.15.64:51404) 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 -- 172.21.15.64:0/2263859675 learned_addr learned my addr 172.21.15.64:0/2263859675 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fd525640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fe527640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f8161ad0 0x7f52f81f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 -- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 msgr2=0x7f52f80b9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80b9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 -- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f8161ad0 msgr2=0x7f52f81f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f8161ad0 0x7f52f81f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 -- 172.21.15.64:0/2263859675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f81f4030 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fdd26640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52f000ec10 tx=0x7f52f000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0017070 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f000cc30 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f81f4310 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0013470 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.931+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f80b99b0 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.932+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f80ccd20 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.933+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f52f001a070 con 0x7f52f80b4fb0 2024-04-01T00:13:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.933+0000 7f52f67fc640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 0x7f52e406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.933+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f52f008aa20 con 0x7f52f80b4fb0 2024-04-01T00:13:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.933+0000 7f52fd525640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 0x7f52e406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.934+0000 7f52fd525640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 0x7f52e406de50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52ec007f70 tx=0x7f52ec007f00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:58.937+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f52f0059070 con 0x7f52f80b4fb0 2024-04-01T00:13:59.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.119+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52f80681a0 con 0x7f52f80b4fb0 2024-04-01T00:13:59.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.119+0000 7f52f67fc640 1 -- 172.21.15.64:0/2263859675 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f52f005d110 con 0x7f52f80b4fb0 2024-04-01T00:13:59.120 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 msgr2=0x7f52e406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 0x7f52e406de50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52ec007f70 tx=0x7f52ec007f00 comp rx=0 tx=0).stop 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 msgr2=0x7f52f81f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52f000ec10 tx=0x7f52f000c520 comp rx=0 tx=0).stop 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 shutdown_connections 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f52e406b990 0x7f52e406de50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f8161ad0 0x7f52f81f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52f81616f0 0x7f52f80b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 --2- 172.21.15.64:0/2263859675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f80b4fb0 0x7f52f81f7970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.121+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 >> 172.21.15.64:0/2263859675 conn(0x7f52f80d3f60 msgr2=0x7f52f80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.122+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 shutdown_connections 2024-04-01T00:13:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.122+0000 7f52fff8a640 1 -- 172.21.15.64:0/2263859675 wait complete. 2024-04-01T00:13:59.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:13:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.237+0000 7f22e04d4640 1 Processor -- start 2024-04-01T00:13:59.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.237+0000 7f22e04d4640 1 -- start start 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22e04d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f22d815e640 0x7f22d80d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22e04d4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f22d80b2f00 0x7f22d80ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22e04d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d80cf400 con 0x7f22d815e260 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22e04d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f22d80d11d0 con 0x7f22d815e640 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22e04d4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f22d80d0b80 con 0x7f22d80b2f00 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dda6f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f22d815e640 0x7f22d80d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22de270640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f22d815e260 0x7f22d80ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48072/0 (socket says 172.21.15.64:48072) 2024-04-01T00:13:59.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22de270640 1 -- 172.21.15.64:0/4085228525 learned_addr learned my addr 172.21.15.64:0/4085228525 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dea71640 1 --2- 172.21.15.64:0/4085228525 >> v2:172.21.15.100:3300/0 conn(0x7f22d80b2f00 0x7f22d80ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dda6f640 1 -- 172.21.15.64:0/4085228525 >> v2:172.21.15.100:3300/0 conn(0x7f22d80b2f00 msgr2=0x7f22d80ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dda6f640 1 --2- 172.21.15.64:0/4085228525 >> v2:172.21.15.100:3300/0 conn(0x7f22d80b2f00 0x7f22d80ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dda6f640 1 -- 172.21.15.64:0/4085228525 >> v1:172.21.15.64:6789/0 conn(0x7f22d815e260 legacy=0x7f22d80ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dd26e640 1 -- 172.21.15.64:0/4085228525 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829973093 0 0) 0x7f22d80cf400 con 0x7f22d815e260 2024-04-01T00:13:59.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.238+0000 7f22dda6f640 1 -- 172.21.15.64:0/4085228525 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d80d0e60 con 0x7f22d815e640 2024-04-01T00:13:59.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22dda6f640 1 --2- 172.21.15.64:0/4085228525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e640 0x7f22d80d2890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22d806e390 tx=0x7f22c802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713240b6efa61641 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22dd26e640 1 -- 172.21.15.64:0/4085228525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8034070 con 0x7f22d815e640 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22dd26e640 1 -- 172.21.15.64:0/4085228525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c8030610 con 0x7f22d815e640 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22dd26e640 1 -- 172.21.15.64:0/4085228525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80309e0 con 0x7f22d815e640 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22e04d4640 1 -- 172.21.15.64:0/4085228525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e640 msgr2=0x7f22d80d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.239+0000 7f22e04d4640 1 --2- 172.21.15.64:0/4085228525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e640 0x7f22d80d2890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22d806e390 tx=0x7f22c802b960 comp rx=0 tx=0).stop 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 -- 172.21.15.64:0/4085228525 shutdown_connections 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 --2- 172.21.15.64:0/4085228525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e640 0x7f22d80d2890 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 --2- 172.21.15.64:0/4085228525 >> v2:172.21.15.100:3300/0 conn(0x7f22d80b2f00 0x7f22d80ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 -- 172.21.15.64:0/4085228525 >> 172.21.15.64:0/4085228525 conn(0x7f22d80d1d70 msgr2=0x7f22d80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 -- 172.21.15.64:0/4085228525 shutdown_connections 2024-04-01T00:13:59.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.240+0000 7f22e04d4640 1 -- 172.21.15.64:0/4085228525 wait complete. 2024-04-01T00:13:59.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 Processor -- start 2024-04-01T00:13:59.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 -- start start 2024-04-01T00:13:59.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22d80b2f00 0x7f22d815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d815e640 0x7f22d8160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80cc940 con 0x7f22d815e260 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80cc7c0 con 0x7f22d815e640 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22e04d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f22d80ccac0 con 0x7f22d80b2f00 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22dda6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22dda6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51416/0 (socket says 172.21.15.64:51416) 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.241+0000 7f22dda6f640 1 -- 172.21.15.64:0/1778380120 learned_addr learned my addr 172.21.15.64:0/1778380120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dea71640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d815e640 0x7f22d8160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22de270640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22d80b2f00 0x7f22d815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 -- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22d80b2f00 msgr2=0x7f22d815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22d80b2f00 0x7f22d815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 -- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d815e640 msgr2=0x7f22d8160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d815e640 0x7f22d8160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 -- 172.21.15.64:0/1778380120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d80cb9d0 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22dda6f640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22c802b9a0 tx=0x7f22c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8047070 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c8030410 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d81f3c00 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8034040 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.242+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81f4190 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.243+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d80681a0 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.244+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f22c804a070 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.244+0000 7f22ceffd640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 0x7f22bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.244+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22c80abed0 con 0x7f22d815e260 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.244+0000 7f22de270640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 0x7f22bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.244+0000 7f22de270640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 0x7f22bc06de50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000f60 tx=0x7f22d4007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.248+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f22d80cb9d0 con 0x7f22d815e260 2024-04-01T00:13:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.426+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22d806d580 con 0x7f22d815e260 2024-04-01T00:13:59.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.427+0000 7f22ceffd640 1 -- 172.21.15.64:0/1778380120 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f22c807a490 con 0x7f22d815e260 2024-04-01T00:13:59.428 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 msgr2=0x7f22bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 0x7f22bc06de50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000f60 tx=0x7f22d4007d50 comp rx=0 tx=0).stop 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 msgr2=0x7f22d80ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22c802b9a0 tx=0x7f22c80046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 shutdown_connections 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f22bc06b990 0x7f22bc06de50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d815e640 0x7f22d8160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22d815e260 0x7f22d80ca600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 --2- 172.21.15.64:0/1778380120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22d80b2f00 0x7f22d815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 >> 172.21.15.64:0/1778380120 conn(0x7f22d80d1d70 msgr2=0x7f22d80c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 shutdown_connections 2024-04-01T00:13:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.429+0000 7f22e04d4640 1 -- 172.21.15.64:0/1778380120 wait complete. 2024-04-01T00:13:59.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:13:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.540+0000 7fd5883a7640 1 Processor -- start 2024-04-01T00:13:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.541+0000 7fd5883a7640 1 -- start start 2024-04-01T00:13:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.541+0000 7fd5883a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.541+0000 7fd5883a7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5800d0060 con 0x7fd58006caa0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.541+0000 7fd5883a7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58006d100 con 0x7fd5800cc340 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.541+0000 7fd5883a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800d0650 con 0x7fd58015e1c0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586944640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd58006caa0 0x7fd5800b0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48078/0 (socket says 172.21.15.64:48078) 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586944640 1 -- 172.21.15.64:0/3452199901 learned_addr learned my addr 172.21.15.64:0/3452199901 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586143640 1 --2- 172.21.15.64:0/3452199901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 261173405 0 0) 0x7fd578001490 con 0x7fd58006caa0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd56c003520 con 0x7fd58006caa0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203368195 0 0) 0x7fd58006d100 con 0x7fd5800cc340 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586143640 1 -- 172.21.15.64:0/3452199901 >> v1:172.21.15.100:6789/0 conn(0x7fd5800cc340 legacy=0x7fd58006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586143640 1 -- 172.21.15.64:0/3452199901 >> v1:172.21.15.64:6789/0 conn(0x7fd58006caa0 legacy=0x7fd5800b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.542+0000 7fd586143640 1 -- 172.21.15.64:0/3452199901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58006d3e0 con 0x7fd58015e1c0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd586143640 1 --2- 172.21.15.64:0/3452199901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800d1c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097d0 tx=0x7fd57002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75f707f382b412d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035020 con 0x7fd58015e1c0 2024-04-01T00:13:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570031800 con 0x7fd58015e1c0 2024-04-01T00:13:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd585141640 1 -- 172.21.15.64:0/3452199901 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570031bb0 con 0x7fd58015e1c0 2024-04-01T00:13:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 -- 172.21.15.64:0/3452199901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 msgr2=0x7fd5800d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 --2- 172.21.15.64:0/3452199901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800d1c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097d0 tx=0x7fd57002b8a0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 -- 172.21.15.64:0/3452199901 shutdown_connections 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 --2- 172.21.15.64:0/3452199901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800d1c10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 -- 172.21.15.64:0/3452199901 >> 172.21.15.64:0/3452199901 conn(0x7fd58009a980 msgr2=0x7fd58009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 -- 172.21.15.64:0/3452199901 shutdown_connections 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.543+0000 7fd5883a7640 1 -- 172.21.15.64:0/3452199901 wait complete. 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 Processor -- start 2024-04-01T00:13:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 -- start start 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5800cc340 0x7fd5800b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800d22d0 con 0x7fd58006caa0 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800d2150 con 0x7fd5800cc340 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd5883a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800d2450 con 0x7fd58015e1c0 2024-04-01T00:13:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd586143640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd586143640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51430/0 (socket says 172.21.15.64:51430) 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd586143640 1 -- 172.21.15.64:0/2550200306 learned_addr learned my addr 172.21.15.64:0/2550200306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd586944640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.545+0000 7fd585942640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5800cc340 0x7fd5800b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 -- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 msgr2=0x7fd5800b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 -- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5800cc340 msgr2=0x7fd5800b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5800cc340 0x7fd5800b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 -- 172.21.15.64:0/2550200306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5800aef20 con 0x7fd58006caa0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd586143640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd570000c00 tx=0x7fd5700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570006e10 con 0x7fd58006caa0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57003bd70 con 0x7fd58006caa0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570045650 con 0x7fd58006caa0 2024-04-01T00:13:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5800b1230 con 0x7fd58006caa0 2024-04-01T00:13:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.546+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5800b17f0 con 0x7fd58006caa0 2024-04-01T00:13:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.547+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58015e6c0 con 0x7fd58006caa0 2024-04-01T00:13:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.548+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd57004c060 con 0x7fd58006caa0 2024-04-01T00:13:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.548+0000 7fd576ffd640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 0x7fd56406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.548+0000 7fd585942640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 0x7fd56406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.548+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5700ac610 con 0x7fd58006caa0 2024-04-01T00:13:59.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.549+0000 7fd585942640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 0x7fd56406de50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5800b0140 tx=0x7fd57c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.553+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd570040070 con 0x7fd58006caa0 2024-04-01T00:13:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.770+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5800681a0 con 0x7fd58006caa0 2024-04-01T00:13:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.770+0000 7fd576ffd640 1 -- 172.21.15.64:0/2550200306 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fd57007ad80 con 0x7fd58006caa0 2024-04-01T00:13:59.771 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:13:59.772 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:13:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:13:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 msgr2=0x7fd56406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 0x7fd56406de50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5800b0140 tx=0x7fd57c009040 comp rx=0 tx=0).stop 2024-04-01T00:13:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 msgr2=0x7fd5800ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd570000c00 tx=0x7fd5700046c0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 shutdown_connections 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56406b990 0x7fd56406de50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58015e1c0 0x7fd5800b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5800cc340 0x7fd5800b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 --2- 172.21.15.64:0/2550200306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58006caa0 0x7fd5800ae9e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 >> 172.21.15.64:0/2550200306 conn(0x7fd58009a980 msgr2=0x7fd5800d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 shutdown_connections 2024-04-01T00:13:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.772+0000 7fd5883a7640 1 -- 172.21.15.64:0/2550200306 wait complete. 2024-04-01T00:13:59.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:13:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.885+0000 7f824d00f640 1 Processor -- start 2024-04-01T00:13:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 -- start start 2024-04-01T00:13:59.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 0x7f82480d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f82480b3820 0x7f82480cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480b3c80 con 0x7f82480cf910 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f82480d2ba0 con 0x7f82480b3820 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824d00f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f82480d2e80 con 0x7f82481619e0 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824759e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f82480cf910 0x7f82480d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48086/0 (socket says 172.21.15.64:48086) 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.886+0000 7f824759e640 1 -- 172.21.15.64:0/3998916338 learned_addr learned my addr 172.21.15.64:0/3998916338 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f823ffff640 1 --2- 172.21.15.64:0/3998916338 >> v2:172.21.15.100:3300/0 conn(0x7f82480b3820 0x7f82480cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f8246d9d640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 0x7f82480d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f824659c640 1 -- 172.21.15.64:0/3998916338 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589591532 0 0) 0x7f82480b3c80 con 0x7f82480cf910 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f824659c640 1 -- 172.21.15.64:0/3998916338 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8234003520 con 0x7f82480cf910 2024-04-01T00:13:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f823ffff640 1 -- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 msgr2=0x7f82480d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f823ffff640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 0x7f82480d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f823ffff640 1 -- 172.21.15.64:0/3998916338 >> v1:172.21.15.64:6789/0 conn(0x7f82480cf910 legacy=0x7f82480d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f823ffff640 1 -- 172.21.15.64:0/3998916338 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82480ceeb0 con 0x7f82480b3820 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.887+0000 7f8246d9d640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 0x7f82480d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f823ffff640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480b3820 0x7f82480cf3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82300097a0 tx=0x7f823002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d454c898c343493 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824659c640 1 -- 172.21.15.64:0/3998916338 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82300315a0 con 0x7f82480b3820 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824659c640 1 -- 172.21.15.64:0/3998916338 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8230037070 con 0x7f82480b3820 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824659c640 1 -- 172.21.15.64:0/3998916338 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230031960 con 0x7f82480b3820 2024-04-01T00:13:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 -- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480b3820 msgr2=0x7f82480cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480b3820 0x7f82480cf3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82300097a0 tx=0x7f823002b840 comp rx=0 tx=0).stop 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 -- 172.21.15.64:0/3998916338 shutdown_connections 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480b3820 0x7f82480cf3d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 --2- 172.21.15.64:0/3998916338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82481619e0 0x7f82480d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 -- 172.21.15.64:0/3998916338 >> 172.21.15.64:0/3998916338 conn(0x7f824809d290 msgr2=0x7f82480d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 -- 172.21.15.64:0/3998916338 shutdown_connections 2024-04-01T00:13:59.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.888+0000 7f824d00f640 1 -- 172.21.15.64:0/3998916338 wait complete. 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 Processor -- start 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 -- start start 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82480b3820 0x7f824814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480cf910 0x7f82480a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82480b31b0 con 0x7f82481619e0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82480d2890 con 0x7f82480cf910 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.890+0000 7f824d00f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f82480ceb10 con 0x7f82480b3820 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51432/0 (socket says 172.21.15.64:51432) 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 -- 172.21.15.64:0/916970988 learned_addr learned my addr 172.21.15.64:0/916970988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f823ffff640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480cf910 0x7f82480a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f8246d9d640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82480b3820 0x7f824814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 -- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82480b3820 msgr2=0x7f824814f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82480b3820 0x7f824814f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 -- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480cf910 msgr2=0x7f82480a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480cf910 0x7f82480a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 -- 172.21.15.64:0/916970988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82480b8b20 con 0x7f82481619e0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824759e640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f824000cb40 tx=0x7f824000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240017070 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824000aca0 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824814fa20 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240012620 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.891+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82480b9380 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.892+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82480b8ed0 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.892+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f824001a070 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.893+0000 7f823f7fe640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 0x7f822806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.893+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f824008a620 con 0x7f82481619e0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.893+0000 7f8246d9d640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 0x7f822806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:13:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.893+0000 7f8246d9d640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 0x7f822806de50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8238009560 tx=0x7f8238009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:13:59.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:13:59.897+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8240058be0 con 0x7f82481619e0 2024-04-01T00:14:00.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.078+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f824809d730 con 0x7f82481619e0 2024-04-01T00:14:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.079+0000 7f823f7fe640 1 -- 172.21.15.64:0/916970988 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f824005cc80 con 0x7f82481619e0 2024-04-01T00:14:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:00.080 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:00.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 msgr2=0x7f822806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 0x7f822806de50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8238009560 tx=0x7f8238009190 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 msgr2=0x7f82480b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f824000cb40 tx=0x7f824000a060 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 shutdown_connections 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f822806b990 0x7f822806de50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82481619e0 0x7f82480b4940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82480cf910 0x7f82480a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 --2- 172.21.15.64:0/916970988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82480b3820 0x7f824814f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 >> 172.21.15.64:0/916970988 conn(0x7f824809d290 msgr2=0x7f82480b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 shutdown_connections 2024-04-01T00:14:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.081+0000 7f824d00f640 1 -- 172.21.15.64:0/916970988 wait complete. 2024-04-01T00:14:00.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:00.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.197+0000 7f06c60ed640 1 Processor -- start 2024-04-01T00:14:00.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 -- start start 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06c00b2df0 0x7f06c00ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c00b52c0 con 0x7f06c0161a50 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f06c00b5070 con 0x7f06c00b2df0 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06c60ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06c00d4bf0 con 0x7f06c00d0f20 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.198+0000 7f06beffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06c00b2df0 0x7f06c00ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06c00b2df0 0x7f06c00ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51436/0 (socket says 172.21.15.64:51436) 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 -- 172.21.15.64:0/3287540056 learned_addr learned my addr 172.21.15.64:0/3287540056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06bffff640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 -- 172.21.15.64:0/3287540056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 msgr2=0x7f06c00cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 -- 172.21.15.64:0/3287540056 >> v1:172.21.15.100:6789/0 conn(0x7f06c0161a50 legacy=0x7f06c00d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06beffd640 1 -- 172.21.15.64:0/3287540056 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c00b55a0 con 0x7f06c00b2df0 2024-04-01T00:14:00.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.199+0000 7f06be7fc640 1 -- 172.21.15.64:0/3287540056 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553679713 0 0) 0x7f06c00b52c0 con 0x7f06c0161a50 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06beffd640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c00b2df0 0x7f06c00ceae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009870 tx=0x7f06b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4bd3fb1b67757a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06be7fc640 1 -- 172.21.15.64:0/3287540056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0035020 con 0x7f06c00b2df0 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06be7fc640 1 -- 172.21.15.64:0/3287540056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b0031640 con 0x7f06c00b2df0 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06be7fc640 1 -- 172.21.15.64:0/3287540056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b002d040 con 0x7f06c00b2df0 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 -- 172.21.15.64:0/3287540056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c00b2df0 msgr2=0x7f06c00ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c00b2df0 0x7f06c00ceae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009870 tx=0x7f06b002b8f0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 -- 172.21.15.64:0/3287540056 shutdown_connections 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c00b2df0 0x7f06c00ceae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 --2- 172.21.15.64:0/3287540056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 -- 172.21.15.64:0/3287540056 >> 172.21.15.64:0/3287540056 conn(0x7f06c009d1c0 msgr2=0x7f06c00d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 -- 172.21.15.64:0/3287540056 shutdown_connections 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.200+0000 7f06c60ed640 1 -- 172.21.15.64:0/3287540056 wait complete. 2024-04-01T00:14:00.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.201+0000 7f06c60ed640 1 Processor -- start 2024-04-01T00:14:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 -- start start 2024-04-01T00:14:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c00b2df0 0x7f06c0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06c00b4950 con 0x7f06c0161a50 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06c00d4880 con 0x7f06c00b2df0 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06c60ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06c00b4ad0 con 0x7f06c00d0f20 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06bffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06bffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51442/0 (socket says 172.21.15.64:51442) 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06bffff640 1 -- 172.21.15.64:0/4099283747 learned_addr learned my addr 172.21.15.64:0/4099283747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06beffd640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.202+0000 7f06bf7fe640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c00b2df0 0x7f06c0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 -- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 msgr2=0x7f06c00ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 -- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c00b2df0 msgr2=0x7f06c0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c00b2df0 0x7f06c0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 -- 172.21.15.64:0/4099283747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c01f7700 con 0x7f06c0161a50 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06bffff640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009e30 tx=0x7f06b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8018020 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b8014480 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8014870 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c01f79e0 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.203+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01f7fa0 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.204+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c00681a0 con 0x7f06c0161a50 2024-04-01T00:14:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.205+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f06b8005d10 con 0x7f06c0161a50 2024-04-01T00:14:00.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.205+0000 7f06b7fff640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 0x7f06a006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.205+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06b808b5a0 con 0x7f06c0161a50 2024-04-01T00:14:00.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.205+0000 7f06bf7fe640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 0x7f06a006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.206+0000 7f06bf7fe640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 0x7f06a006de50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000f60 tx=0x7f06a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.209+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f06b8022030 con 0x7f06c0161a50 2024-04-01T00:14:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.389+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06c00d2b60 con 0x7f06c0161a50 2024-04-01T00:14:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.389+0000 7f06b7fff640 1 -- 172.21.15.64:0/4099283747 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f06b8010040 con 0x7f06c0161a50 2024-04-01T00:14:00.390 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 msgr2=0x7f06a006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 0x7f06a006de50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000f60 tx=0x7f06a8009040 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 msgr2=0x7f06c006caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009e30 tx=0x7f06b800c490 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 shutdown_connections 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006b990 0x7f06a006de50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c0161a50 0x7f06c006caa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06c00d0f20 0x7f06c00ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 --2- 172.21.15.64:0/4099283747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06c00b2df0 0x7f06c0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 >> 172.21.15.64:0/4099283747 conn(0x7f06c009d1c0 msgr2=0x7f06c01605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 shutdown_connections 2024-04-01T00:14:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.391+0000 7f06c60ed640 1 -- 172.21.15.64:0/4099283747 wait complete. 2024-04-01T00:14:00.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:00.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.507+0000 7f959d54f640 1 Processor -- start 2024-04-01T00:14:00.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.507+0000 7f959d54f640 1 -- start start 2024-04-01T00:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f959d54f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95980d24b0 0x7f95980b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f959d54f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 0x7f959806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f959d54f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95980b1f10 con 0x7f95980cf2c0 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f959d54f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f95980b21f0 con 0x7f95980d24b0 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f959d54f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f95980b24d0 con 0x7f95980cee60 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95980d24b0 0x7f95980b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95980d24b0 0x7f95980b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51454/0 (socket says 172.21.15.64:51454) 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 -- 172.21.15.64:0/3775649022 learned_addr learned my addr 172.21.15.64:0/3775649022 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f958e7fc640 1 --2- 172.21.15.64:0/3775649022 >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 0x7f959806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 -- 172.21.15.64:0/3775649022 >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 msgr2=0x7f959806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 --2- 172.21.15.64:0/3775649022 >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 0x7f959806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 -- 172.21.15.64:0/3775649022 >> v1:172.21.15.100:6789/0 conn(0x7f95980cf2c0 legacy=0x7f959815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f9596ffd640 1 -- 172.21.15.64:0/3775649022 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95980b27b0 con 0x7f95980d24b0 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.508+0000 7f958e7fc640 1 --2- 172.21.15.64:0/3775649022 >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 0x7f959806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f9596ffd640 1 --2- 172.21.15.64:0/3775649022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980d24b0 0x7f95980b1930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f959806e840 tx=0x7f958802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f430549851464a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f95967fc640 1 -- 172.21.15.64:0/3775649022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588034070 con 0x7f95980d24b0 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f95967fc640 1 -- 172.21.15.64:0/3775649022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588030860 con 0x7f95980d24b0 2024-04-01T00:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f95967fc640 1 -- 172.21.15.64:0/3775649022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588030c30 con 0x7f95980d24b0 2024-04-01T00:14:00.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f959d54f640 1 -- 172.21.15.64:0/3775649022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980d24b0 msgr2=0x7f95980b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.509+0000 7f959d54f640 1 --2- 172.21.15.64:0/3775649022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980d24b0 0x7f95980b1930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f959806e840 tx=0x7f958802b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 -- 172.21.15.64:0/3775649022 shutdown_connections 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 --2- 172.21.15.64:0/3775649022 >> v2:172.21.15.100:3300/0 conn(0x7f95980cee60 0x7f959806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 --2- 172.21.15.64:0/3775649022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980d24b0 0x7f95980b1930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 -- 172.21.15.64:0/3775649022 >> 172.21.15.64:0/3775649022 conn(0x7f959809a980 msgr2=0x7f959809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 -- 172.21.15.64:0/3775649022 shutdown_connections 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.510+0000 7f959d54f640 1 -- 172.21.15.64:0/3775649022 wait complete. 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.511+0000 7f959d54f640 1 Processor -- start 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.511+0000 7f959d54f640 1 -- start start 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95980cf2c0 0x7f95980afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95980d24b0 0x7f959815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f959806d120 con 0x7f95980cee60 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95980d2890 con 0x7f95980d24b0 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f959806d2a0 con 0x7f95980cf2c0 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51462/0 (socket says 172.21.15.64:51462) 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 -- 172.21.15.64:0/1018841072 learned_addr learned my addr 172.21.15.64:0/1018841072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f958e7fc640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95980cf2c0 0x7f95980afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f95977fe640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95980d24b0 0x7f959815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 -- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95980cf2c0 msgr2=0x7f95980afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95980cf2c0 0x7f95980afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 -- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95980d24b0 msgr2=0x7f959815fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95980d24b0 0x7f959815fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 -- 172.21.15.64:0/1018841072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95980c6060 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f9596ffd640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f958802b9e0 tx=0x7f95880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588034040 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588039d90 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95980b0280 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588043630 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.512+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95980b0840 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.513+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95980681a0 con 0x7f95980cee60 2024-04-01T00:14:00.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.514+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f958804a060 con 0x7f95980cee60 2024-04-01T00:14:00.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.514+0000 7f958ffff640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 0x7f957806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.514+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f95880ac6e0 con 0x7f95980cee60 2024-04-01T00:14:00.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.514+0000 7f958e7fc640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 0x7f957806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.515+0000 7f958e7fc640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 0x7f957806de50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9580009560 tx=0x7f9580009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.518+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f958803e070 con 0x7f95980cee60 2024-04-01T00:14:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.697+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95980b1710 con 0x7f95980cee60 2024-04-01T00:14:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.697+0000 7f958ffff640 1 -- 172.21.15.64:0/1018841072 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f958807ad30 con 0x7f95980cee60 2024-04-01T00:14:00.698 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 msgr2=0x7f957806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 0x7f957806de50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9580009560 tx=0x7f9580009190 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 msgr2=0x7f95981f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f958802b9e0 tx=0x7f95880046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 shutdown_connections 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f957806b990 0x7f957806de50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95980d24b0 0x7f959815fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95980cf2c0 0x7f95980afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 --2- 172.21.15.64:0/1018841072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95980cee60 0x7f95981f36a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.699+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 >> 172.21.15.64:0/1018841072 conn(0x7f959809a980 msgr2=0x7f9598070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.700+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 shutdown_connections 2024-04-01T00:14:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.700+0000 7f959d54f640 1 -- 172.21.15.64:0/1018841072 wait complete. 2024-04-01T00:14:00.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.815+0000 7f61e6f19640 1 Processor -- start 2024-04-01T00:14:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.815+0000 7f61e6f19640 1 -- start start 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e6f19640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 0x7f61e00cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e6f19640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61e015e7c0 0x7f61e00d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e6f19640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e00cee70 con 0x7f61e00d15e0 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e6f19640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f61e00cf150 con 0x7f61e015e7c0 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e6f19640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f61e00cf430 con 0x7f61e015e3e0 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61e015e7c0 0x7f61e00d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61e015e7c0 0x7f61e00d4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51464/0 (socket says 172.21.15.64:51464) 2024-04-01T00:14:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 -- 172.21.15.64:0/3569507949 learned_addr learned my addr 172.21.15.64:0/3569507949 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e4cb5640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 0x7f61e00cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61df7fe640 1 -- 172.21.15.64:0/3569507949 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600963543 0 0) 0x7f61e00cee70 con 0x7f61e00d15e0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61df7fe640 1 -- 172.21.15.64:0/3569507949 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c8003520 con 0x7f61e00d15e0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 -- 172.21.15.64:0/3569507949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 msgr2=0x7f61e00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 0x7f61e00cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 -- 172.21.15.64:0/3569507949 >> v1:172.21.15.100:6789/0 conn(0x7f61e00d15e0 legacy=0x7f61e00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61dffff640 1 -- 172.21.15.64:0/3569507949 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e00cf710 con 0x7f61e015e7c0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.816+0000 7f61e4cb5640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 0x7f61e00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.817+0000 7f61dffff640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e015e7c0 0x7f61e00d4720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009870 tx=0x7f61d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dad221bac809030 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.817+0000 7f61df7fe640 1 -- 172.21.15.64:0/3569507949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4034070 con 0x7f61e015e7c0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.817+0000 7f61df7fe640 1 -- 172.21.15.64:0/3569507949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4030840 con 0x7f61e015e7c0 2024-04-01T00:14:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.817+0000 7f61df7fe640 1 -- 172.21.15.64:0/3569507949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4030c10 con 0x7f61e015e7c0 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 -- 172.21.15.64:0/3569507949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e015e7c0 msgr2=0x7f61e00d4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e015e7c0 0x7f61e00d4720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009870 tx=0x7f61d402b980 comp rx=0 tx=0).stop 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 -- 172.21.15.64:0/3569507949 shutdown_connections 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e015e7c0 0x7f61e00d4720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 --2- 172.21.15.64:0/3569507949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e3e0 0x7f61e00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 -- 172.21.15.64:0/3569507949 >> 172.21.15.64:0/3569507949 conn(0x7f61e009d210 msgr2=0x7f61e00d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 -- 172.21.15.64:0/3569507949 shutdown_connections 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.818+0000 7f61e6f19640 1 -- 172.21.15.64:0/3569507949 wait complete. 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.819+0000 7f61e6f19640 1 Processor -- start 2024-04-01T00:14:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.819+0000 7f61e6f19640 1 -- start start 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61e015e3e0 0x7f61e01533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e7c0 0x7f61e00c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00cec50 con 0x7f61e00d15e0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00cead0 con 0x7f61e015e3e0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e6f19640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f61e00d4c60 con 0x7f61e015e7c0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51480/0 (socket says 172.21.15.64:51480) 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 -- 172.21.15.64:0/1320834017 learned_addr learned my addr 172.21.15.64:0/1320834017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e54b6640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e7c0 0x7f61e00c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61dffff640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61e015e3e0 0x7f61e01533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 -- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e7c0 msgr2=0x7f61e00c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e7c0 0x7f61e00c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 -- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61e015e3e0 msgr2=0x7f61e01533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61e015e3e0 0x7f61e01533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.820+0000 7f61e4cb5640 1 -- 172.21.15.64:0/1320834017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e00ca5c0 con 0x7f61e00d15e0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61e4cb5640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f60 tx=0x7f61d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8018020 con 0x7f61e00d15e0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8014680 con 0x7f61e00d15e0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8014a50 con 0x7f61e00d15e0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e00c92c0 con 0x7f61e00d15e0 2024-04-01T00:14:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.821+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e00c9880 con 0x7f61e00d15e0 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.822+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e00681a0 con 0x7f61e00d15e0 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.822+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f61d8005d10 con 0x7f61e00d15e0 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.823+0000 7f61dd7fa640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 0x7f61cc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.823+0000 7f61dffff640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 0x7f61cc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.823+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61d808b490 con 0x7f61e00d15e0 2024-04-01T00:14:00.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.823+0000 7f61dffff640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 0x7f61cc06de50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61d4036000 tx=0x7f61d403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:00.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:00.827+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f61d8022030 con 0x7f61e00d15e0 2024-04-01T00:14:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.007+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61e009d720 con 0x7f61e00d15e0 2024-04-01T00:14:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.007+0000 7f61dd7fa640 1 -- 172.21.15.64:0/1320834017 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f61d8010040 con 0x7f61e00d15e0 2024-04-01T00:14:01.009 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 msgr2=0x7f61cc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 0x7f61cc06de50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61d4036000 tx=0x7f61d403b040 comp rx=0 tx=0).stop 2024-04-01T00:14:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 msgr2=0x7f61e00ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009f60 tx=0x7f61d8009f90 comp rx=0 tx=0).stop 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 shutdown_connections 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f61cc06b990 0x7f61cc06de50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61e015e7c0 0x7f61e00c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61e015e3e0 0x7f61e01533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 --2- 172.21.15.64:0/1320834017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e00d15e0 0x7f61e00ca080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 >> 172.21.15.64:0/1320834017 conn(0x7f61e009d210 msgr2=0x7f61e006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 shutdown_connections 2024-04-01T00:14:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.009+0000 7f61e6f19640 1 -- 172.21.15.64:0/1320834017 wait complete. 2024-04-01T00:14:01.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.122+0000 7f61b75b5640 1 Processor -- start 2024-04-01T00:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b75b5640 1 -- start start 2024-04-01T00:14:01.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b75b5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61b006c3e0 0x7f61b00d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b75b5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b00d2780 con 0x7f61b015e3f0 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b75b5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b00ce790 con 0x7f61b00d1920 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b75b5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f61b00cf3a0 con 0x7f61b006c3e0 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b5b52640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61b006c3e0 0x7f61b00d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b4b50640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61b015e3f0 0x7f61b00cff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48100/0 (socket says 172.21.15.64:48100) 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.123+0000 7f61b4b50640 1 -- 172.21.15.64:0/2660399432 learned_addr learned my addr 172.21.15.64:0/2660399432 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.124+0000 7f61b5b52640 1 -- 172.21.15.64:0/2660399432 >> v1:172.21.15.100:6789/0 conn(0x7f61b00d1920 legacy=0x7f61b00d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.124+0000 7f61a7fff640 1 -- 172.21.15.64:0/2660399432 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704730716 0 0) 0x7f61b00ce790 con 0x7f61b00d1920 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.124+0000 7f61b5b52640 1 -- 172.21.15.64:0/2660399432 >> v1:172.21.15.64:6789/0 conn(0x7f61b015e3f0 legacy=0x7f61b00cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.124+0000 7f61b5b52640 1 -- 172.21.15.64:0/2660399432 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b00cea70 con 0x7f61b006c3e0 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.124+0000 7f61b5b52640 1 --2- 172.21.15.64:0/2660399432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00d0610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009d90 tx=0x7f61ac02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dd156f2be3566e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61a7fff640 1 -- 172.21.15.64:0/2660399432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac039070 con 0x7f61b006c3e0 2024-04-01T00:14:01.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61a7fff640 1 -- 172.21.15.64:0/2660399432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ac02cab0 con 0x7f61b006c3e0 2024-04-01T00:14:01.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61a7fff640 1 -- 172.21.15.64:0/2660399432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac034440 con 0x7f61b006c3e0 2024-04-01T00:14:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 -- 172.21.15.64:0/2660399432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 msgr2=0x7f61b00d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 --2- 172.21.15.64:0/2660399432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00d0610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009d90 tx=0x7f61ac02bd80 comp rx=0 tx=0).stop 2024-04-01T00:14:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 -- 172.21.15.64:0/2660399432 shutdown_connections 2024-04-01T00:14:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 --2- 172.21.15.64:0/2660399432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00d0610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 -- 172.21.15.64:0/2660399432 >> 172.21.15.64:0/2660399432 conn(0x7f61b009a980 msgr2=0x7f61b009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 -- 172.21.15.64:0/2660399432 shutdown_connections 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.125+0000 7f61b75b5640 1 -- 172.21.15.64:0/2660399432 wait complete. 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.126+0000 7f61b75b5640 1 Processor -- start 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.126+0000 7f61b75b5640 1 -- start start 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b00d1920 0x7f61b0161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b01534b0 0x7f61b00b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61b00b0cd0 con 0x7f61b006c3e0 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61b00cf030 con 0x7f61b00d1920 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b75b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f61b00b0e50 con 0x7f61b01534b0 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5351640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5351640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51490/0 (socket says 172.21.15.64:51490) 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5351640 1 -- 172.21.15.64:0/3639138971 learned_addr learned my addr 172.21.15.64:0/3639138971 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5b52640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b01534b0 0x7f61b00b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b4b50640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b00d1920 0x7f61b0161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5351640 1 -- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b01534b0 msgr2=0x7f61b00b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.127+0000 7f61b5351640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b01534b0 0x7f61b00b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b5351640 1 -- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b00d1920 msgr2=0x7f61b0161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b5351640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b00d1920 0x7f61b0161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b5351640 1 -- 172.21.15.64:0/3639138971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0161590 con 0x7f61b006c3e0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b5351640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c4e0 tx=0x7f61a800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8017070 con 0x7f61b006c3e0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a80035f0 con 0x7f61b006c3e0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b00b4f20 con 0x7f61b006c3e0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80039e0 con 0x7f61b006c3e0 2024-04-01T00:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.128+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b00b54e0 con 0x7f61b006c3e0 2024-04-01T00:14:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.129+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00681a0 con 0x7f61b006c3e0 2024-04-01T00:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.129+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f61a801a070 con 0x7f61b006c3e0 2024-04-01T00:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.130+0000 7f61a5ffb640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 0x7f619406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.130+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61a8089ae0 con 0x7f61b006c3e0 2024-04-01T00:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.130+0000 7f61b4b50640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 0x7f619406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.130+0000 7f61b4b50640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 0x7f619406de50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61b01f7140 tx=0x7f61a0007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.134+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f61a80580a0 con 0x7f61b006c3e0 2024-04-01T00:14:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.352+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61b015d910 con 0x7f61b006c3e0 2024-04-01T00:14:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.352+0000 7f61a5ffb640 1 -- 172.21.15.64:0/3639138971 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f61a805c140 con 0x7f61b006c3e0 2024-04-01T00:14:01.353 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:01.354 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:01.354 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:01.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 msgr2=0x7f619406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 0x7f619406de50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61b01f7140 tx=0x7f61a0007e20 comp rx=0 tx=0).stop 2024-04-01T00:14:01.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 msgr2=0x7f61b00b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c4e0 tx=0x7f61a800c9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 shutdown_connections 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f619406b990 0x7f619406de50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b01534b0 0x7f61b00b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b00d1920 0x7f61b0161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 --2- 172.21.15.64:0/3639138971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b006c3e0 0x7f61b00b48d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 >> 172.21.15.64:0/3639138971 conn(0x7f61b009a980 msgr2=0x7f61b00cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 shutdown_connections 2024-04-01T00:14:01.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.354+0000 7f61b75b5640 1 -- 172.21.15.64:0/3639138971 wait complete. 2024-04-01T00:14:01.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.466+0000 7f59b42ed640 1 Processor -- start 2024-04-01T00:14:01.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.466+0000 7f59b42ed640 1 -- start start 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b42ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 0x7f59ac072800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b42ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59ac15b760 0x7f59ac0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b42ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac072d40 con 0x7f59ac0cbce0 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b42ed640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac15e550 con 0x7f59ac15b760 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b42ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac15e830 con 0x7f59ac072420 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b288a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59ac15b760 0x7f59ac0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b288a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59ac15b760 0x7f59ac0cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51500/0 (socket says 172.21.15.64:51500) 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b288a640 1 -- 172.21.15.64:0/372778522 learned_addr learned my addr 172.21.15.64:0/372778522 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.467+0000 7f59b2089640 1 --2- 172.21.15.64:0/372778522 >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 0x7f59ac072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b288a640 1 -- 172.21.15.64:0/372778522 >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 msgr2=0x7f59ac072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b288a640 1 --2- 172.21.15.64:0/372778522 >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 0x7f59ac072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b288a640 1 -- 172.21.15.64:0/372778522 >> v1:172.21.15.100:6789/0 conn(0x7f59ac0cbce0 legacy=0x7f59ac06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b288a640 1 -- 172.21.15.64:0/372778522 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac15eb10 con 0x7f59ac15b760 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b2089640 1 --2- 172.21.15.64:0/372778522 >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 0x7f59ac072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b1087640 1 -- 172.21.15.64:0/372778522 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847970016 0 0) 0x7f59ac072d40 con 0x7f59ac0cbce0 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b288a640 1 --2- 172.21.15.64:0/372778522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac0cde70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b28b0 tx=0x7f599c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1855c96eef079df server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b1087640 1 -- 172.21.15.64:0/372778522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c02ca80 con 0x7f59ac15b760 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b1087640 1 -- 172.21.15.64:0/372778522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c02cc20 con 0x7f59ac15b760 2024-04-01T00:14:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.468+0000 7f59b1087640 1 -- 172.21.15.64:0/372778522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0343f0 con 0x7f59ac15b760 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 -- 172.21.15.64:0/372778522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 msgr2=0x7f59ac0cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 --2- 172.21.15.64:0/372778522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac0cde70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b28b0 tx=0x7f599c02bee0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 -- 172.21.15.64:0/372778522 shutdown_connections 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 --2- 172.21.15.64:0/372778522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac0cde70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 --2- 172.21.15.64:0/372778522 >> v2:172.21.15.100:3300/0 conn(0x7f59ac072420 0x7f59ac072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 -- 172.21.15.64:0/372778522 >> 172.21.15.64:0/372778522 conn(0x7f59ac06b4e0 msgr2=0x7f59ac06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 -- 172.21.15.64:0/372778522 shutdown_connections 2024-04-01T00:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.469+0000 7f59b42ed640 1 -- 172.21.15.64:0/372778522 wait complete. 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.470+0000 7f59b42ed640 1 Processor -- start 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.470+0000 7f59b42ed640 1 -- start start 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 0x7f59ac09a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0cbce0 0x7f59ac09c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0cc140 con 0x7f59ac15b760 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac06eb00 con 0x7f59ac0cbce0 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b42ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0cc2c0 con 0x7f59ac072420 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51502/0 (socket says 172.21.15.64:51502) 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 -- 172.21.15.64:0/4056850993 learned_addr learned my addr 172.21.15.64:0/4056850993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b2089640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 0x7f59ac09a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b1888640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0cbce0 0x7f59ac09c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 -- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 msgr2=0x7f59ac09a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 0x7f59ac09a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 -- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0cbce0 msgr2=0x7f59ac09c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0cbce0 0x7f59ac09c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b288a640 1 -- 172.21.15.64:0/4056850993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac09aec0 con 0x7f59ac15b760 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.471+0000 7f59b2089640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 0x7f59ac09a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59b288a640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f599c009f40 tx=0x7f599c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c043030 con 0x7f59ac15b760 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c03e450 con 0x7f59ac15b760 2024-04-01T00:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac0afad0 con 0x7f59ac15b760 2024-04-01T00:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c009510 con 0x7f59ac15b760 2024-04-01T00:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.472+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac0b0090 con 0x7f59ac15b760 2024-04-01T00:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.473+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac0681a0 con 0x7f59ac15b760 2024-04-01T00:14:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.473+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f599c046070 con 0x7f59ac15b760 2024-04-01T00:14:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.473+0000 7f59aaffd640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 0x7f599806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.474+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f599c0ac9a0 con 0x7f59ac15b760 2024-04-01T00:14:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.474+0000 7f59b2089640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 0x7f599806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.474+0000 7f59b2089640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 0x7f599806de50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009560 tx=0x7f59a4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.478+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f599c04d070 con 0x7f59ac15b760 2024-04-01T00:14:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.659+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59ac0b0430 con 0x7f59ac15b760 2024-04-01T00:14:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.660+0000 7f59aaffd640 1 -- 172.21.15.64:0/4056850993 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f599c07b110 con 0x7f59ac15b760 2024-04-01T00:14:01.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:01.661 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 msgr2=0x7f599806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 0x7f599806de50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009560 tx=0x7f59a4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 msgr2=0x7f59ac14c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f599c009f40 tx=0x7f599c004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 shutdown_connections 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f599806b990 0x7f599806de50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59ac15b760 0x7f59ac14c790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0cbce0 0x7f59ac09c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 --2- 172.21.15.64:0/4056850993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac072420 0x7f59ac09a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 >> 172.21.15.64:0/4056850993 conn(0x7f59ac06b4e0 msgr2=0x7f59ac06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 shutdown_connections 2024-04-01T00:14:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.662+0000 7f59b42ed640 1 -- 172.21.15.64:0/4056850993 wait complete. 2024-04-01T00:14:01.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:01.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.774+0000 7fe1ba716640 1 Processor -- start 2024-04-01T00:14:01.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.774+0000 7fe1ba716640 1 -- start start 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1ba716640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 0x7fe1b40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1ba716640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b415e550 con 0x7fe1b415b760 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1ba716640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b415e830 con 0x7fe1b4072420 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1ba716640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b415eb10 con 0x7fe1b406e3f0 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b37fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe1b415b760 0x7fe1b40cbce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48114/0 (socket says 172.21.15.64:48114) 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b37fe640 1 -- 172.21.15.64:0/1422284092 learned_addr learned my addr 172.21.15.64:0/1422284092 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b8cb3640 1 --2- 172.21.15.64:0/1422284092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 0x7fe1b40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858901861 0 0) 0x7fe1b415e550 con 0x7fe1b415b760 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1a0003520 con 0x7fe1b415b760 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 665172056 0 0) 0x7fe1b415e830 con 0x7fe1b4072420 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.775+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b415e550 con 0x7fe1b4072420 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.776+0000 7fe1b8cb3640 1 -- 172.21.15.64:0/1422284092 >> v1:172.21.15.100:6789/0 conn(0x7fe1b4072420 legacy=0x7fe1b4072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.776+0000 7fe1b8cb3640 1 -- 172.21.15.64:0/1422284092 >> v1:172.21.15.64:6789/0 conn(0x7fe1b415b760 legacy=0x7fe1b40cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.776+0000 7fe1b8cb3640 1 -- 172.21.15.64:0/1422284092 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b415edf0 con 0x7fe1b406e3f0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.776+0000 7fe1b8cb3640 1 --2- 172.21.15.64:0/1422284092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 0x7fe1b40cde70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009dc0 tx=0x7fe1ac02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e58d16998e0175 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.776+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac039070 con 0x7fe1b406e3f0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac02cc40 con 0x7fe1b406e3f0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1422284092 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac034440 con 0x7fe1b406e3f0 2024-04-01T00:14:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 -- 172.21.15.64:0/1422284092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 msgr2=0x7fe1b40cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 --2- 172.21.15.64:0/1422284092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 0x7fe1b40cde70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009dc0 tx=0x7fe1ac02be90 comp rx=0 tx=0).stop 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 -- 172.21.15.64:0/1422284092 shutdown_connections 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 --2- 172.21.15.64:0/1422284092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b406e3f0 0x7fe1b40cde70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 -- 172.21.15.64:0/1422284092 >> 172.21.15.64:0/1422284092 conn(0x7fe1b406b4e0 msgr2=0x7fe1b406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 -- 172.21.15.64:0/1422284092 shutdown_connections 2024-04-01T00:14:01.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.777+0000 7fe1ba716640 1 -- 172.21.15.64:0/1422284092 wait complete. 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.778+0000 7fe1ba716640 1 Processor -- start 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.778+0000 7fe1ba716640 1 -- start start 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1b4072420 0x7fe1b414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b415b760 0x7fe1b40b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b406e880 con 0x7fe1b406e3f0 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b415e330 con 0x7fe1b4072420 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1ba716640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b406ea00 con 0x7fe1b415b760 2024-04-01T00:14:01.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51516/0 (socket says 172.21.15.64:51516) 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 -- 172.21.15.64:0/673262129 learned_addr learned my addr 172.21.15.64:0/673262129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b37fe640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1b4072420 0x7fe1b414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b8cb3640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b415b760 0x7fe1b40b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 -- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b415b760 msgr2=0x7fe1b40b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b415b760 0x7fe1b40b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 -- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1b4072420 msgr2=0x7fe1b414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1b4072420 0x7fe1b414d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.779+0000 7fe1b3fff640 1 -- 172.21.15.64:0/673262129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b41683a0 con 0x7fe1b406e3f0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1b3fff640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c490 tx=0x7fe19c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c0052d0 con 0x7fe1b406e3f0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19c005470 con 0x7fe1b406e3f0 2024-04-01T00:14:01.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b4168680 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c012850 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.780+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b4168c40 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.781+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b40681a0 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.781+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe19c0129f0 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.782+0000 7fe1b0ff9640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 0x7fe19406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.782+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe19c08a400 con 0x7fe1b406e3f0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.782+0000 7fe1b37fe640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 0x7fe19406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:01.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.782+0000 7fe1b37fe640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 0x7fe19406de50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe1b41f54c0 tx=0x7fe1a4007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:01.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:01.786+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe19c0589d0 con 0x7fe1b406e3f0 2024-04-01T00:14:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.003+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1b4168e80 con 0x7fe1b406e3f0 2024-04-01T00:14:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.003+0000 7fe1b0ff9640 1 -- 172.21.15.64:0/673262129 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fe1b4168e80 con 0x7fe1b406e3f0 2024-04-01T00:14:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:02.005 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 msgr2=0x7fe19406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 0x7fe19406de50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe1b41f54c0 tx=0x7fe1a4007d30 comp rx=0 tx=0).stop 2024-04-01T00:14:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 msgr2=0x7fe1b409aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c490 tx=0x7fe19c00c950 comp rx=0 tx=0).stop 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 shutdown_connections 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe19406b990 0x7fe19406de50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1b415b760 0x7fe1b40b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1b4072420 0x7fe1b414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 --2- 172.21.15.64:0/673262129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1b406e3f0 0x7fe1b409aca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 >> 172.21.15.64:0/673262129 conn(0x7fe1b406b4e0 msgr2=0x7fe1b406b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 shutdown_connections 2024-04-01T00:14:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.005+0000 7fe1ba716640 1 -- 172.21.15.64:0/673262129 wait complete. 2024-04-01T00:14:02.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-01T00:13:46.387702+0000', 'modified': '2024-04-01T00:13:48.671456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/2705635055', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2705635055}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2705635055}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4272, 'qdb_cluster': [4272]} 2024-04-01T00:14:02.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-01T00:14:02.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-01T00:14:02.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.524+0000 7eff08864640 1 Processor -- start 2024-04-01T00:14:02.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.524+0000 7eff08864640 1 -- start start 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff08864640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff0406cb00 0x7eff040cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff08864640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 0x7eff04167d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff08864640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff040b1ce0 con 0x7eff0406cee0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff08864640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7eff040cf4d0 con 0x7eff0406cb00 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff08864640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff0415e480 con 0x7eff0415dff0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7efefbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff0406cee0 0x7eff0415d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48118/0 (socket says 172.21.15.64:48118) 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7efefbfff640 1 -- 172.21.15.64:0/280396037 learned_addr learned my addr 172.21.15.64:0/280396037 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff02d9d640 1 --2- 172.21.15.64:0/280396037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 0x7eff04167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff0259c640 1 --2- 172.21.15.64:0/280396037 >> v2:172.21.15.100:3300/0 conn(0x7eff0406cb00 0x7eff040cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff01d9b640 1 -- 172.21.15.64:0/280396037 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316931358 0 0) 0x7eff040b1ce0 con 0x7eff0406cee0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff01d9b640 1 -- 172.21.15.64:0/280396037 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efef0003520 con 0x7eff0406cee0 2024-04-01T00:14:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff02d9d640 1 -- 172.21.15.64:0/280396037 >> v2:172.21.15.100:3300/0 conn(0x7eff0406cb00 msgr2=0x7eff040cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff02d9d640 1 --2- 172.21.15.64:0/280396037 >> v2:172.21.15.100:3300/0 conn(0x7eff0406cb00 0x7eff040cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff02d9d640 1 -- 172.21.15.64:0/280396037 >> v1:172.21.15.64:6789/0 conn(0x7eff0406cee0 legacy=0x7eff0415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.525+0000 7eff02d9d640 1 -- 172.21.15.64:0/280396037 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040b1310 con 0x7eff0415dff0 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.526+0000 7eff02d9d640 1 --2- 172.21.15.64:0/280396037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 0x7eff04167d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d90 tx=0x7efefc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efff928327dd2b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.526+0000 7eff01d9b640 1 -- 172.21.15.64:0/280396037 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc039070 con 0x7eff0415dff0 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.526+0000 7eff01d9b640 1 -- 172.21.15.64:0/280396037 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc02cbc0 con 0x7eff0415dff0 2024-04-01T00:14:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.526+0000 7eff01d9b640 1 -- 172.21.15.64:0/280396037 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc034440 con 0x7eff0415dff0 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 -- 172.21.15.64:0/280396037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 msgr2=0x7eff04167d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 --2- 172.21.15.64:0/280396037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 0x7eff04167d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d90 tx=0x7efefc02be10 comp rx=0 tx=0).stop 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 -- 172.21.15.64:0/280396037 shutdown_connections 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 --2- 172.21.15.64:0/280396037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0415dff0 0x7eff04167d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 --2- 172.21.15.64:0/280396037 >> v2:172.21.15.100:3300/0 conn(0x7eff0406cb00 0x7eff040cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 -- 172.21.15.64:0/280396037 >> 172.21.15.64:0/280396037 conn(0x7eff0409a980 msgr2=0x7eff0409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 -- 172.21.15.64:0/280396037 shutdown_connections 2024-04-01T00:14:02.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.527+0000 7eff08864640 1 -- 172.21.15.64:0/280396037 wait complete. 2024-04-01T00:14:02.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.528+0000 7eff08864640 1 Processor -- start 2024-04-01T00:14:02.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.528+0000 7eff08864640 1 -- start start 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0406cb00 0x7eff0415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 0x7eff040b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff040b2560 con 0x7eff0415dff0 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff040b2280 con 0x7eff0406cee0 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff08864640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff040b1960 con 0x7eff0406cb00 2024-04-01T00:14:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51528/0 (socket says 172.21.15.64:51528) 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 -- 172.21.15.64:0/2365601087 learned_addr learned my addr 172.21.15.64:0/2365601087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7efefbfff640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 0x7eff040b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff0259c640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0406cb00 0x7eff0415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 -- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0406cb00 msgr2=0x7eff0415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0406cb00 0x7eff0415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 -- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 msgr2=0x7eff040b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 0x7eff040b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 -- 172.21.15.64:0/2365601087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff041f75a0 con 0x7eff0415dff0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7efefbfff640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 0x7eff040b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efefc037040 tx=0x7efefc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.529+0000 7efefaffd640 1 -- 172.21.15.64:0/2365601087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc039050 con 0x7eff0415dff0 2024-04-01T00:14:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.530+0000 7efefaffd640 1 -- 172.21.15.64:0/2365601087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc03a040 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.530+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff041f7880 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.530+0000 7efefaffd640 1 -- 172.21.15.64:0/2365601087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc009450 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.530+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041f7e40 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.531+0000 7efefaffd640 1 -- 172.21.15.64:0/2365601087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7efefc047020 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.531+0000 7efefaffd640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 0x7efee406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.531+0000 7efefaffd640 1 -- 172.21.15.64:0/2365601087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efefc0ac660 con 0x7eff0415dff0 2024-04-01T00:14:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.532+0000 7eff0259c640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 0x7efee406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.532+0000 7eff0259c640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 0x7efee406de50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efef4009aa0 tx=0x7efef4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.532+0000 7eff0b444740 1 Processor -- start 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.532+0000 7eff0b444740 1 -- start start 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196796d0 0x55a519679ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a51967d720 0x55a51967f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5195e2af0 con 0x55a5196a9460 2024-04-01T00:14:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a51958b9d0 con 0x55a51967d720 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55a5195deb90 con 0x55a5196796d0 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51530/0 (socket says 172.21.15.64:51530) 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 -- 172.21.15.64:0/2143511266 learned_addr learned my addr 172.21.15.64:0/2143511266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7efefbfff640 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a51967d720 0x55a51967f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff0259c640 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196796d0 0x55a519679ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 -- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196796d0 msgr2=0x55a519679ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196796d0 0x55a519679ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 -- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a51967d720 msgr2=0x55a51967f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a51967d720 0x55a51967f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.533+0000 7eff02d9d640 1 -- 172.21.15.64:0/2143511266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5192f72d0 con 0x55a5196a9460 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efefc031040 tx=0x7efefc07a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9305412a2fdfb67d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7efee3fff640 1 -- 172.21.15.64:0/2143511266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc083640 con 0x55a5196a9460 2024-04-01T00:14:02.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7efee3fff640 1 -- 172.21.15.64:0/2143511266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc003e10 con 0x55a5196a9460 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7efee3fff640 1 -- 172.21.15.64:0/2143511266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc0773e0 con 0x55a5196a9460 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 -- 172.21.15.64:0/2143511266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 msgr2=0x55a5196a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efefc031040 tx=0x7efefc07a9e0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 -- 172.21.15.64:0/2143511266 shutdown_connections 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196a9460 0x55a5196a98e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a51967d720 0x55a51967f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 --2- 172.21.15.64:0/2143511266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196796d0 0x55a519679ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 -- 172.21.15.64:0/2143511266 >> 172.21.15.64:0/2143511266 conn(0x55a51967b750 msgr2=0x55a51967d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 -- 172.21.15.64:0/2143511266 shutdown_connections 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.534+0000 7eff0b444740 1 -- 172.21.15.64:0/2143511266 wait complete. 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.535+0000 7eff0b444740 1 -- start start 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.535+0000 7eff0b444740 1 Processor -- start 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a5196c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a51968fc50 0x55a5196aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5195e2af0 con 0x55a5196ad250 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a51959ef70 con 0x55a5196a9460 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55a51967dbb0 con 0x55a51968fc50 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0b444740 10 client.0 _ll_register_callbacks cb 0x55a5192f6ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51542/0 (socket says 172.21.15.64:51542) 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 -- 172.21.15.64:0/1090487060 learned_addr learned my addr 172.21.15.64:0/1090487060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff0259c640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a5196c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7eff02d9d640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a51968fc50 0x55a5196aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 -- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a51968fc50 msgr2=0x55a5196aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a51968fc50 0x55a5196aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 -- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 msgr2=0x55a5196c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a5196c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 -- 172.21.15.64:0/1090487060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5193c16b0 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efefbfff640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efeec0127f0 tx=0x7efeec012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adc1cc6065b5917 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.536+0000 7efee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec019020 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efeec0058b0 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efefc003e10 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efeec023050 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 10 client.4641 fetch_fsmap learned FSMap version 7 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55a5196c76c0 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc003e10 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7efeec026020 con 0x55a5196ad250 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 10 client.4641 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 20 client.4641 populate_metadata read hostname 'smithi064' 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a5196a59c0 0x55a5196e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 4 client.4641 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 --> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5195f0030 con 0x55a5196a59c0 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.537+0000 7eff0259c640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a5196a59c0 0x55a5196e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7efee17fa640 10 client.4641 ms_handle_connect on v2:172.21.15.100:6834/2705635055 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7eff0259c640 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a5196a59c0 0x55a5196e3000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748c55fc8092d048 server_cookie=138f6ccf8c23f832 in_seq=0 out_seq=0 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7efee17fa640 1 -- 172.21.15.64:0/1090487060 <== mds.0 v2:172.21.15.100:6834/2705635055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a5195f0030 con 0x55a5196a59c0 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7efee17fa640 10 client.4641 handle_command_reply: tid=0 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7eff0b444740 1 client.4641 shutdown 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 msgr2=0x55a51969dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.538+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efeec0127f0 tx=0x7efeec012cb0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 shutdown_connections 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a51968fc50 0x55a5196aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5196ad250 0x55a51969dcc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a5196a59c0 0x55a5196e3000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 --2- 172.21.15.64:0/1090487060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a5196c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7efee17fa640 0 client.4641 ms_handle_reset on v2:172.21.15.100:6834/2705635055 2024-04-01T00:14:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 >> 172.21.15.64:0/1090487060 conn(0x55a5196796d0 msgr2=0x55a5196c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 shutdown_connections 2024-04-01T00:14:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 1 -- 172.21.15.64:0/1090487060 wait complete. 2024-04-01T00:14:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.539+0000 7eff0b444740 20 client.4641 trim_cache size 0 max 16384 2024-04-01T00:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.557+0000 7eff0b444740 1 Processor -- start 2024-04-01T00:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.557+0000 7eff0b444740 1 -- start start 2024-04-01T00:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196a9460 0x55a5196ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 0x55a51969dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff04160360 con 0x55a51968fc50 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff040d1b20 con 0x55a5196ad250 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff0415fd10 con 0x55a5196a9460 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51544/0 (socket says 172.21.15.64:51544) 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 -- 172.21.15.64:0/2215295331 learned_addr learned my addr 172.21.15.64:0/2215295331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7efefbfff640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196a9460 0x55a5196ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 0x55a51969dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 -- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196a9460 msgr2=0x55a5196ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196a9460 0x55a5196ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 -- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 msgr2=0x55a51969dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 0x55a51969dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 -- 172.21.15.64:0/2215295331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5196a2a90 con 0x55a51968fc50 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff02d9d640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 0x55a51969dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7eff0259c640 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efef4009520 tx=0x7efef4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05d45ca75b03241 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7efee37fe640 1 -- 172.21.15.64:0/2215295331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef402db10 con 0x55a51968fc50 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7efee37fe640 1 -- 172.21.15.64:0/2215295331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4032070 con 0x55a51968fc50 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.558+0000 7efee37fe640 1 -- 172.21.15.64:0/2215295331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef402d4b0 con 0x55a51968fc50 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- 172.21.15.64:0/2215295331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 msgr2=0x55a5196aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efef4009520 tx=0x7efef4019000 comp rx=0 tx=0).stop 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- 172.21.15.64:0/2215295331 shutdown_connections 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196ad250 0x55a51969dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196a9460 0x55a5196ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 --2- 172.21.15.64:0/2215295331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196aa5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- 172.21.15.64:0/2215295331 >> 172.21.15.64:0/2215295331 conn(0x55a5196796d0 msgr2=0x55a519679ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- 172.21.15.64:0/2215295331 shutdown_connections 2024-04-01T00:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- 172.21.15.64:0/2215295331 wait complete. 2024-04-01T00:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 -- start start 2024-04-01T00:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.559+0000 7eff0b444740 1 Processor -- start 2024-04-01T00:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a519678340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196ad250 0x55a51967f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5196c8450 con 0x55a51968fc50 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a5196c79a0 con 0x55a5196a9460 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55a51969e2d0 con 0x55a5196ad250 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0b444740 10 client.0 _ll_register_callbacks cb 0x7efeec001770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51554/0 (socket says 172.21.15.64:51554) 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 -- 172.21.15.64:0/1038120487 learned_addr learned my addr 172.21.15.64:0/1038120487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7efefbfff640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a519678340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff02d9d640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196ad250 0x55a51967f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 -- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196ad250 msgr2=0x55a51967f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196ad250 0x55a51967f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 -- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 msgr2=0x55a519678340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a519678340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.560+0000 7eff0259c640 1 -- 172.21.15.64:0/1038120487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5196a1a70 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0259c640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efef4012010 tx=0x7efef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79109168f93487c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef403a020 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efef4005990 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef4032070 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efef4044050 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 10 client.4647 fetch_fsmap learned FSMap version 7 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55a519678880 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4032070 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7efef4047020 con 0x55a51968fc50 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 10 client.4647 fetch_fsmap finished waiting for FSMap version 7 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 20 client.4647 populate_metadata read hostname 'smithi064' 2024-04-01T00:14:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a51973eb60 0x55a5196a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 4 client.4647 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 --> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] -- command(tid 0: {"prefix": "status"}) -- 0x55a5195ea380 con 0x55a51973eb60 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.561+0000 7efefbfff640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a51973eb60 0x55a5196a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7efee17fa640 10 client.4647 ms_handle_connect on v2:172.21.15.100:6834/2705635055 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7efefbfff640 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a51973eb60 0x55a5196a8240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ccc09f5dc3b2d5 server_cookie=89269f5ee8ba06b0 in_seq=0 out_seq=0 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7efee17fa640 1 -- 172.21.15.64:0/1038120487 <== mds.0 v2:172.21.15.100:6834/2705635055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x55a5195ea380 con 0x55a51973eb60 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7efee17fa640 10 client.4647 handle_command_reply: tid=0 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 client.4647 shutdown 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 msgr2=0x55a5196a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efef4012010 tx=0x7efef4004730 comp rx=0 tx=0).stop 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 shutdown_connections 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a5196ad250 0x55a51967f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:6834/2705635055,v1:172.21.15.100:6835/2705635055] conn(0x55a51973eb60 0x55a5196a8240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5196a9460 0x55a519678340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 --2- 172.21.15.64:0/1038120487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51968fc50 0x55a5196a68a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7efee17fa640 0 client.4647 ms_handle_reset on v2:172.21.15.100:6834/2705635055 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 >> 172.21.15.64:0/1038120487 conn(0x55a5196796d0 msgr2=0x55a5196ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 shutdown_connections 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 1 -- 172.21.15.64:0/1038120487 wait complete. 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.562+0000 7eff0b444740 20 client.4647 trim_cache size 0 max 16384 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "05d5f08b-c3c0-4c90-b927-5e55cbe6aed5", 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4272, 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-04-01T00:14:02.571 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 15.540398261, 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 7, 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 20, 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 20, 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 22.23839899 2024-04-01T00:14:02.572 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 msgr2=0x7efee406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 0x7efee406de50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efef4009aa0 tx=0x7efef4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 msgr2=0x7eff040b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efefc037040 tx=0x7efefc004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 shutdown_connections 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efee406b990 0x7efee406de50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff0415dff0 0x7eff040b7310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff0406cee0 0x7eff040b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 --2- 172.21.15.64:0/2365601087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff0406cb00 0x7eff0415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 >> 172.21.15.64:0/2365601087 conn(0x7eff0409a980 msgr2=0x7eff040af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 shutdown_connections 2024-04-01T00:14:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.563+0000 7eff08864640 1 -- 172.21.15.64:0/2365601087 wait complete. 2024-04-01T00:14:02.576 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-01T00:14:02.600 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-01T00:14:02.600 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-01T00:14:02.600 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:02.601 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:02.601 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:02.601 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-01T00:14:02.601 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:02.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:02.602 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-01T00:14:02.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:02.619 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-01T00:14:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-04-01T00:14:02.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:14:02.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:02.690 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-04-01T00:14:02.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:02.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-04-01T00:14:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-04-01T00:14:02.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:14:02.778 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-01T00:14:02.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-01T00:14:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.918+0000 7f1948d0d640 1 Processor -- start 2024-04-01T00:14:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.918+0000 7f1948d0d640 1 -- start start 2024-04-01T00:14:02.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.918+0000 7f1948d0d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19440d0650 0x7f19440b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1948d0d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1944070eb0 0x7f19440cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1948d0d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194415fde0 con 0x7f19440723d0 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1948d0d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f19441600c0 con 0x7f19440d0650 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1948d0d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f19441603a0 con 0x7f1944070eb0 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19440d0650 0x7f19440b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1944070eb0 0x7f19440cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f194259c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f19440723d0 0x7f19440cff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:50494/0 (socket says 172.21.15.64:50494) 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19440d0650 0x7f19440b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:45362/0 (socket says 172.21.15.64:45362) 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f194259c640 1 -- 172.21.15.64:0/2247767469 learned_addr learned my addr 172.21.15.64:0/2247767469 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 -- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1944070eb0 msgr2=0x7f19440cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 --2- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1944070eb0 0x7f19440cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 -- 172.21.15.64:0/2247767469 >> v1:172.21.15.100:6789/0 conn(0x7f19440723d0 legacy=0x7f19440cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1939d9b640 1 -- 172.21.15.64:0/2247767469 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1944160680 con 0x7f19440d0650 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.919+0000 7f1941d9b640 1 -- 172.21.15.64:0/2247767469 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571324354 0 0) 0x7f194415fde0 con 0x7f19440723d0 2024-04-01T00:14:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1939d9b640 1 --2- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19440d0650 0x7f19440b3820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f192c0097a0 tx=0x7f192c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=750511ab6b84a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1941d9b640 1 -- 172.21.15.64:0/2247767469 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0315a0 con 0x7f19440d0650 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1941d9b640 1 -- 172.21.15.64:0/2247767469 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c037070 con 0x7f19440d0650 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1941d9b640 1 -- 172.21.15.64:0/2247767469 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c031960 con 0x7f19440d0650 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 -- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19440d0650 msgr2=0x7f19440b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 --2- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19440d0650 0x7f19440b3820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f192c0097a0 tx=0x7f192c02b840 comp rx=0 tx=0).stop 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 -- 172.21.15.64:0/2247767469 shutdown_connections 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 --2- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19440d0650 0x7f19440b3820 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 --2- 172.21.15.64:0/2247767469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1944070eb0 0x7f19440cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 -- 172.21.15.64:0/2247767469 >> 172.21.15.64:0/2247767469 conn(0x7f194406b4e0 msgr2=0x7f194406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.920+0000 7f1948d0d640 1 -- 172.21.15.64:0/2247767469 shutdown_connections 2024-04-01T00:14:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.921+0000 7f1948d0d640 1 -- 172.21.15.64:0/2247767469 wait complete. 2024-04-01T00:14:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.922+0000 7f1948d0d640 1 Processor -- start 2024-04-01T00:14:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.922+0000 7f1948d0d640 1 -- start start 2024-04-01T00:14:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 0x7f194415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19440723d0 0x7f19441506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f19440718a0 con 0x7f19440d0650 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1944071720 con 0x7f1944070eb0 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f19440727b0 con 0x7f19440723d0 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51568/0 (socket says 172.21.15.64:51568) 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f194259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 0x7f194415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 -- 172.21.15.64:0/213948287 learned_addr learned my addr 172.21.15.64:0/213948287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1939d9b640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19440723d0 0x7f19441506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 -- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19440723d0 msgr2=0x7f19441506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19440723d0 0x7f19441506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 -- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 msgr2=0x7f194415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 0x7f194415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 -- 172.21.15.64:0/213948287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194415cf40 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f194259c640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 0x7f194415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1942d9d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f193c00efc0 tx=0x7f193c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c018020 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c014480 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194415d220 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c014870 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.923+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194415d7e0 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.925+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19440681a0 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.925+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f193c005d10 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.925+0000 7f193b7fe640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 0x7f192406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.925+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f193c08b130 con 0x7f19440d0650 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.925+0000 7f194259c640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 0x7f192406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.926+0000 7f194259c640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 0x7f192406de50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1934000f60 tx=0x7f1934007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:02.930+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f193c022070 con 0x7f19440d0650 2024-04-01T00:14:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.150+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19440ce250 con 0x7f19440d0650 2024-04-01T00:14:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.151+0000 7f193b7fe640 1 -- 172.21.15.64:0/213948287 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f193c010040 con 0x7f19440d0650 2024-04-01T00:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 msgr2=0x7f192406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 0x7f192406de50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1934000f60 tx=0x7f1934007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 msgr2=0x7f194415c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f193c00efc0 tx=0x7f193c00c490 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 shutdown_connections 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f192406b990 0x7f192406de50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440d0650 0x7f194415c480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19440723d0 0x7f19441506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 --2- 172.21.15.64:0/213948287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1944070eb0 0x7f194415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.152+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 >> 172.21.15.64:0/213948287 conn(0x7f194406b4e0 msgr2=0x7f19440b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.153+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 shutdown_connections 2024-04-01T00:14:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03.153+0000 7f1948d0d640 1 -- 172.21.15.64:0/213948287 wait complete. 2024-04-01T00:14:03.165 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-01T00:14:03.165 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-01T00:14:03.165 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-04-01T00:14:03.165 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-01T00:14:03.165 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:03.166 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-01T00:14:03.166 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-01T00:14:03.166 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-01T00:14:03.166 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-01T00:14:03.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:03.166 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-04-01T00:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42749sec preferred_lft 42749sec 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-04-01T00:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:03.183 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-01T00:14:03.183 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:03.183 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:03.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-04-01T00:14:03.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-04-01T00:14:03.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-04-01T00:14:03.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-01T00:14:03.184 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:03.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:03.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:03.388 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-01T00:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-04-01T00:14:03.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:03.416 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-04-01T00:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-04-01T00:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-01T00:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-01T00:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-01T00:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-01T00:14:03.484 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:03.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:03.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:03.797 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-04-01T00:14:03.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:03.812 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-04-01T00:14:03.866 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:03.866 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:03.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:03.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-01T00:14:03.867 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:03.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:03.990 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-01T00:14:03.990 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:04.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:04.129 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:04.129 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:04.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-04-01T00:14:04.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-04-01T00:14:04.129 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:04.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:04.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:04.211 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-01T00:14:04.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:04.211 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-01T00:14:04.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:04.226 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:04.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:04.348 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:04.402 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:04.403 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:04.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:04.405 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:04.405 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:04.405 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:04.405 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:04.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:04.405 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:04.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-01T00:14:04.457 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-01T00:14:04.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:04.526 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:04.581 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:04.582 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:04.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:04.584 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:04.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-04-01T00:14:04.584+0000 7f9c0b04d480 -1 init, newargv = 0x560eccfc8d90 newargc=15 2024-04-01T00:14:04.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[46890]: starting ceph client 2024-04-01T00:14:04.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[46890]: starting fuse 2024-04-01T00:14:04.600 INFO:teuthology.orchestra.run.smithi064.stdout:41 2024-04-01T00:14:04.600 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-04-01T00:14:04.601 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-04-01T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-04-01T00:14:04.722 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.46890.asok 2024-04-01T00:14:04.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:04.732 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46890.asok 2024-04-01T00:14:04.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:04.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46890.asok status 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "384f31f3d3aee7f8ba3f60846b8844c049165583", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-1977-g384f31f3 (384f31f3d3aee7f8ba3f60846b8844c049165583) squid (dev)", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "46890", 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-04-01T00:14:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4659, 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4659 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2602988068 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2602988068 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4659 192.168.0.1:0/2602988068", 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2602988068", 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 7, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 20, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-04-01T00:14:04.873 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:04.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-01T00:14:04.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.979+0000 7f0ab2148640 1 Processor -- start 2024-04-01T00:14:04.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.979+0000 7f0ab2148640 1 -- start start 2024-04-01T00:14:04.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0ab2148640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aac0b4350 0x7f0aac06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0ab2148640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 0x7f0aac06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0ab2148640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aac0b5d20 con 0x7f0aac147420 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0ab2148640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac161e10 con 0x7f0aac0b4350 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0ab2148640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac1620b0 con 0x7f0aac06ee70 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aac0b4350 0x7f0aac06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aac0b4350 0x7f0aac06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52710/0 (socket says 172.21.15.64:52710) 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 -- 172.21.15.64:0/1573178361 learned_addr learned my addr 172.21.15.64:0/1573178361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aabfff640 1 --2- 172.21.15.64:0/1573178361 >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 0x7f0aac06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 -- 172.21.15.64:0/1573178361 >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 msgr2=0x7f0aac06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 --2- 172.21.15.64:0/1573178361 >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 0x7f0aac06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 -- 172.21.15.64:0/1573178361 >> v1:172.21.15.100:6789/0 conn(0x7f0aac147420 legacy=0x7f0aac071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aab7fe640 1 -- 172.21.15.64:0/1573178361 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac162390 con 0x7f0aac0b4350 2024-04-01T00:14:04.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.980+0000 7f0aabfff640 1 --2- 172.21.15.64:0/1573178361 >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 0x7f0aac06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:04.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.981+0000 7f0aab7fe640 1 --2- 172.21.15.64:0/1573178361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac06b0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0d1c00 tx=0x7f0a9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968c0183a9416220 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:04.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.981+0000 7f0aaa7fc640 1 -- 172.21.15.64:0/1573178361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c034070 con 0x7f0aac0b4350 2024-04-01T00:14:04.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.981+0000 7f0aaa7fc640 1 -- 172.21.15.64:0/1573178361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c030860 con 0x7f0aac0b4350 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.981+0000 7f0aaa7fc640 1 -- 172.21.15.64:0/1573178361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c030c30 con 0x7f0aac0b4350 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 -- 172.21.15.64:0/1573178361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 msgr2=0x7f0aac06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1573178361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac06b0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0d1c00 tx=0x7f0a9c02b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 -- 172.21.15.64:0/1573178361 shutdown_connections 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1573178361 >> v2:172.21.15.100:3300/0 conn(0x7f0aac06ee70 0x7f0aac06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1573178361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac06b0c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 -- 172.21.15.64:0/1573178361 >> 172.21.15.64:0/1573178361 conn(0x7f0aac0b2760 msgr2=0x7f0aac0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 -- 172.21.15.64:0/1573178361 shutdown_connections 2024-04-01T00:14:04.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.982+0000 7f0ab2148640 1 -- 172.21.15.64:0/1573178361 wait complete. 2024-04-01T00:14:04.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.984+0000 7f0ab2148640 1 Processor -- start 2024-04-01T00:14:04.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.984+0000 7f0ab2148640 1 -- start start 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aac06ee70 0x7f0aac140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac147420 0x7f0aac1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac147970 con 0x7f0aac0b4350 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac162950 con 0x7f0aac147420 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0ab2148640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac147af0 con 0x7f0aac06ee70 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52714/0 (socket says 172.21.15.64:52714) 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 -- 172.21.15.64:0/1126376789 learned_addr learned my addr 172.21.15.64:0/1126376789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aab7fe640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aac06ee70 0x7f0aac140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aabfff640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac147420 0x7f0aac1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 -- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aac06ee70 msgr2=0x7f0aac140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aac06ee70 0x7f0aac140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 -- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac147420 msgr2=0x7f0aac1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac147420 0x7f0aac1f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 -- 172.21.15.64:0/1126376789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac13eff0 con 0x7f0aac0b4350 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.985+0000 7f0aaaffd640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c3c0 tx=0x7f0aa000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.986+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00052d0 con 0x7f0aac0b4350 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.986+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa0005490 con 0x7f0aac0b4350 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.986+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aac13f2d0 con 0x7f0aac0b4350 2024-04-01T00:14:04.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.986+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0007260 con 0x7f0aac0b4350 2024-04-01T00:14:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.986+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aac1f45a0 con 0x7f0aac0b4350 2024-04-01T00:14:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.987+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aac1f40b0 con 0x7f0aac0b4350 2024-04-01T00:14:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.987+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0aa0007400 con 0x7f0aac0b4350 2024-04-01T00:14:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.988+0000 7f0a93fff640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 0x7f0a9806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.988+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0aa0089d70 con 0x7f0aac0b4350 2024-04-01T00:14:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.988+0000 7f0aab7fe640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 0x7f0a9806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.988+0000 7f0aab7fe640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 0x7f0a9806de80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c036000 tx=0x7f0a9c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:04.992+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0aa0058220 con 0x7f0aac0b4350 2024-04-01T00:14:05.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.211+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0aac0681a0 con 0x7f0aac0b4350 2024-04-01T00:14:05.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.211+0000 7f0a93fff640 1 -- 172.21.15.64:0/1126376789 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f0aa005c2c0 con 0x7f0aac0b4350 2024-04-01T00:14:05.213 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-01T00:14:05.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 msgr2=0x7f0a9806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:05.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 0x7f0a9806de80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c036000 tx=0x7f0a9c008030 comp rx=0 tx=0).stop 2024-04-01T00:14:05.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 msgr2=0x7f0aac13b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:05.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c3c0 tx=0x7f0aa000c880 comp rx=0 tx=0).stop 2024-04-01T00:14:05.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 shutdown_connections 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a9806b9c0 0x7f0a9806de80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac147420 0x7f0aac1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aac0b4350 0x7f0aac13b9e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 --2- 172.21.15.64:0/1126376789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aac06ee70 0x7f0aac140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 >> 172.21.15.64:0/1126376789 conn(0x7f0aac0b2760 msgr2=0x7f0aac067160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 shutdown_connections 2024-04-01T00:14:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05.213+0000 7f0ab2148640 1 -- 172.21.15.64:0/1126376789 wait complete. 2024-04-01T00:14:05.225 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-01T00:14:05.226 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-01T00:14:05.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.227 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-04-01T00:14:05.242 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:05.242 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:05.242 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:05.242 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.242 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42747sec preferred_lft 42747sec 2024-04-01T00:14:05.243 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:26:c8:bb:aa:50 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e041:2bff:fe60:1b0c/64 scope link 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:26:c8:bb:aa:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::f826:c8ff:febb:aa50/64 scope link tentative 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.245 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-01T00:14:05.308 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-04-01T00:14:05.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.309 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-04-01T00:14:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-04-01T00:14:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-01T00:14:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-01T00:14:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-01T00:14:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-01T00:14:05.366 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:05.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.501 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-04-01T00:14:05.516 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-01T00:14:05.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.517 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-04-01T00:14:05.571 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-01T00:14:05.572 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:05.572 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:05.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:05.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-01T00:14:05.572 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:05.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-01T00:14:05.720 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:05.720 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:05.720 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:05.720 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.720 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:e6:15:16:bc:4a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::38e6:15ff:fe16:bc4a/64 scope link tentative 2024-04-01T00:14:05.721 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:05.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-01T00:14:05.789 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:05.789 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:05.789 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:05.789 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.789 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:e6:15:16:bc:4a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::38e6:15ff:fe16:bc4a/64 scope link tentative 2024-04-01T00:14:05.790 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:05.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-01T00:14:05.791 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-01T00:14:05.792 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:05.956 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:05.956 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:05.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2024-04-01T00:14:05.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2024-04-01T00:14:05.956 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:06.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:06.037 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-01T00:14:06.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:06.038 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:06.052 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-01T00:14:06.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:06.053 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:06.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:06.174 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:06.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:06.233 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:06.285 INFO:teuthology.orchestra.run.smithi064.stdout:41 2024-04-01T00:14:06.285 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-01T00:14:06.285 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-01T00:14:06.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:06.354 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:06.409 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:06.409 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:06.409 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:06.409 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:06.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:06.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:06.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:06.412 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:06.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:06.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:06.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:06.413 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:06.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2024-04-01T00:14:06.411+0000 7f930f24c480 -1 init, newargv = 0x56421d0b2dc0 newargc=15 2024-04-01T00:14:06.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[47389]: starting ceph client 2024-04-01T00:14:06.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[47389]: starting fuse 2024-04-01T00:14:06.431 INFO:teuthology.orchestra.run.smithi064.stdout:41 2024-04-01T00:14:06.431 INFO:teuthology.orchestra.run.smithi064.stdout:44 2024-04-01T00:14:06.431 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-04-01T00:14:06.432 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2024-04-01T00:14:06.433 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-04-01T00:14:06.553 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.47389.asok 2024-04-01T00:14:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:06.563 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.47389.asok 2024-04-01T00:14:06.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:06.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.47389.asok status 2024-04-01T00:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "384f31f3d3aee7f8ba3f60846b8844c049165583", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-1977-g384f31f3 (384f31f3d3aee7f8ba3f60846b8844c049165583) squid (dev)", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "47389", 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4668, 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4668 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2129721676 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2129721676 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4668 192.168.0.2:0/2129721676", 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/2129721676", 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 7, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 20, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-04-01T00:14:06.705 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:06.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-01T00:14:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.809+0000 7f2644684640 1 Processor -- start 2024-04-01T00:14:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.809+0000 7f2644684640 1 -- start start 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.810+0000 7f2644684640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f263c0723d0 0x7f263c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.810+0000 7f2644684640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c15fde0 con 0x7f263c070eb0 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.810+0000 7f2644684640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c1600c0 con 0x7f263c0d0650 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.810+0000 7f2644684640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f263c1603a0 con 0x7f263c0723d0 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2642420640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f263c0723d0 0x7f263c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2641c1f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f263c070eb0 0x7f263c0cff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45958/0 (socket says 172.21.15.64:45958) 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2641c1f640 1 -- 172.21.15.64:0/2380879516 learned_addr learned my addr 172.21.15.64:0/2380879516 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2642420640 1 -- 172.21.15.64:0/2380879516 >> v1:172.21.15.100:6789/0 conn(0x7f263c0d0650 legacy=0x7f263c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2642420640 1 -- 172.21.15.64:0/2380879516 >> v1:172.21.15.64:6789/0 conn(0x7f263c070eb0 legacy=0x7f263c0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f264141e640 1 -- 172.21.15.64:0/2380879516 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930761146 0 0) 0x7f263c1600c0 con 0x7f263c0d0650 2024-04-01T00:14:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.811+0000 7f2642420640 1 -- 172.21.15.64:0/2380879516 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c160680 con 0x7f263c0723d0 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2642420640 1 --2- 172.21.15.64:0/2380879516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c0b3820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f263402bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f53440b4a22924 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f264141e640 1 -- 172.21.15.64:0/2380879516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634039070 con 0x7f263c0723d0 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f264141e640 1 -- 172.21.15.64:0/2380879516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263402ca20 con 0x7f263c0723d0 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f264141e640 1 -- 172.21.15.64:0/2380879516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263402cdf0 con 0x7f263c0723d0 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 -- 172.21.15.64:0/2380879516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 msgr2=0x7f263c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 --2- 172.21.15.64:0/2380879516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c0b3820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f263402bcf0 comp rx=0 tx=0).stop 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 -- 172.21.15.64:0/2380879516 shutdown_connections 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 --2- 172.21.15.64:0/2380879516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c0b3820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 -- 172.21.15.64:0/2380879516 >> 172.21.15.64:0/2380879516 conn(0x7f263c06b4e0 msgr2=0x7f263c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 -- 172.21.15.64:0/2380879516 shutdown_connections 2024-04-01T00:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.812+0000 7f2644684640 1 -- 172.21.15.64:0/2380879516 wait complete. 2024-04-01T00:14:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.813+0000 7f2644684640 1 Processor -- start 2024-04-01T00:14:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 -- start start 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c070eb0 0x7f263c0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f263c0d0650 0x7f263c09aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0718a0 con 0x7f263c0723d0 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f263c071720 con 0x7f263c070eb0 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2644684640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f263c0727b0 con 0x7f263c0d0650 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2641c1f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2641c1f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52734/0 (socket says 172.21.15.64:52734) 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2641c1f640 1 -- 172.21.15.64:0/1037398291 learned_addr learned my addr 172.21.15.64:0/1037398291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2642420640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c070eb0 0x7f263c0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.814+0000 7f2642c21640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f263c0d0650 0x7f263c09aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 -- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f263c0d0650 msgr2=0x7f263c09aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f263c0d0650 0x7f263c09aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 -- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c070eb0 msgr2=0x7f263c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c070eb0 0x7f263c0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 -- 172.21.15.64:0/1037398291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c156f20 con 0x7f263c0723d0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2641c1f640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c520 tx=0x7f262c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c017070 con 0x7f263c0723d0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c0074e0 con 0x7f263c0723d0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c004a00 con 0x7f263c0723d0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c157200 con 0x7f263c0723d0 2024-04-01T00:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.815+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c0b06c0 con 0x7f263c0723d0 2024-04-01T00:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.816+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c0681a0 con 0x7f263c0723d0 2024-04-01T00:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.817+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f262c01a070 con 0x7f263c0723d0 2024-04-01T00:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.817+0000 7f2632ffd640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 0x7f262006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.817+0000 7f2642420640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 0x7f262006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.817+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f262c089b90 con 0x7f263c0723d0 2024-04-01T00:14:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.818+0000 7f2642420640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 0x7f262006de50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2634036000 tx=0x7f2634030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:06.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:06.822+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f262c0581e0 con 0x7f263c0723d0 2024-04-01T00:14:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.040+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f263c0ce250 con 0x7f263c0723d0 2024-04-01T00:14:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.041+0000 7f2632ffd640 1 -- 172.21.15.64:0/1037398291 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f262c05c280 con 0x7f263c0723d0 2024-04-01T00:14:07.043 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-01T00:14:07.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 msgr2=0x7f262006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 0x7f262006de50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2634036000 tx=0x7f2634030000 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 msgr2=0x7f263c09a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c520 tx=0x7f262c00c9e0 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 shutdown_connections 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f262006b990 0x7f262006de50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f263c0d0650 0x7f263c09aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0723d0 0x7f263c09a980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 --2- 172.21.15.64:0/1037398291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c070eb0 0x7f263c0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 >> 172.21.15.64:0/1037398291 conn(0x7f263c06b4e0 msgr2=0x7f263c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 shutdown_connections 2024-04-01T00:14:07.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:07.042+0000 7f2644684640 1 -- 172.21.15.64:0/1037398291 wait complete. 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-01T00:14:07.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-01T00:14:07.056 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-01T00:14:07.056 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-01T00:14:07.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:07.056 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-01T00:14:07.072 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:07.073 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-01T00:14:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:07.074 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-01T00:14:07.074 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:07.074 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:07.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-04-01T00:14:07.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-04-01T00:14:07.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-04-01T00:14:07.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-01T00:14:07.075 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:07.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:07.288 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-01T00:14:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-01T00:14:07.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:07.321 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-01T00:14:07.389 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-01T00:14:07.389 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-01T00:14:07.389 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-01T00:14:07.389 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-01T00:14:07.389 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-01T00:14:07.390 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:07.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:07.664 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-01T00:14:07.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:07.679 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-01T00:14:07.733 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:07.733 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:07.733 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:07.734 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-01T00:14:07.734 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:07.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:07.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:07.856 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-01T00:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:07.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:07.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:07.999 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:07.999 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:07.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-01T00:14:07.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-04-01T00:14:07.999 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:08.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:08.083 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-01T00:14:08.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:08.083 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:08.098 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-01T00:14:08.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:08.098 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:08.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-01T00:14:08.221 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:08.277 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-01T00:14:08.277 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-01T00:14:08.278 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-01T00:14:08.279 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:08.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:08.280 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-01T00:14:08.333 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-01T00:14:08.333 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-01T00:14:08.375 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-01T00:14:08.406 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-01T00:14:08.462 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:08.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:08.465 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-01T00:14:08.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:2024-04-01T00:14:08.461+0000 7f5b0500b480 -1 init, newargv = 0x557091541b70 newargc=15 2024-04-01T00:14:08.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[43599]: starting ceph client 2024-04-01T00:14:08.466 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[43599]: starting fuse 2024-04-01T00:14:08.480 INFO:teuthology.orchestra.run.smithi100.stdout:41 2024-04-01T00:14:08.481 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-01T00:14:08.481 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-01T00:14:08.482 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.1")) 2024-04-01T00:14:08.483 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-01T00:14:08.602 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.1.43599.asok 2024-04-01T00:14:08.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:08.612 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43599.asok 2024-04-01T00:14:08.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:08.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43599.asok status 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "384f31f3d3aee7f8ba3f60846b8844c049165583", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1977-g384f31f3 (384f31f3d3aee7f8ba3f60846b8844c049165583) squid (dev)", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "1", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "43599", 2024-04-01T00:14:08.750 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4568, 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4568 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:08.751 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1315171772 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1315171772 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4568 192.168.0.1:0/1315171772", 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/1315171772", 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 7, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 20, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-01T00:14:08.752 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-01T00:14:08.753 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-01T00:14:08.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-01T00:14:08.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.870+0000 7ffac1d93640 1 Processor -- start 2024-04-01T00:14:08.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.870+0000 7ffac1d93640 1 -- start start 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffac1d93640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffabc0d2320 0x7ffabc06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffac1d93640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffabc0d0060 con 0x7ffabc0cc340 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffac1d93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffabc06d100 con 0x7ffabc15e1c0 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffac1d93640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7ffabc0d0650 con 0x7ffabc0d2320 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffabc0d2320 0x7ffabc06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffabc0cc340 0x7ffabc06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45966/0 (socket says 172.21.15.64:45966) 2024-04-01T00:14:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffabc0d2320 0x7ffabc06caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52750/0 (socket says 172.21.15.64:52750) 2024-04-01T00:14:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabaffd640 1 -- 172.21.15.64:0/3392663244 learned_addr learned my addr 172.21.15.64:0/3392663244 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabbfff640 1 -- 172.21.15.64:0/3392663244 >> v1:172.21.15.100:6789/0 conn(0x7ffabc15e1c0 legacy=0x7ffabc0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabbfff640 1 -- 172.21.15.64:0/3392663244 >> v1:172.21.15.64:6789/0 conn(0x7ffabc0cc340 legacy=0x7ffabc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.871+0000 7ffabbfff640 1 -- 172.21.15.64:0/3392663244 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc06d3e0 con 0x7ffabc0d2320 2024-04-01T00:14:08.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.872+0000 7ffabbfff640 1 --2- 172.21.15.64:0/3392663244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0d2320 0x7ffabc06caa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0d07d0 tx=0x7ffab402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92df666f46010cee server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:08.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.872+0000 7ffaba7fc640 1 -- 172.21.15.64:0/3392663244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4039070 con 0x7ffabc0d2320 2024-04-01T00:14:08.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.872+0000 7ffaba7fc640 1 -- 172.21.15.64:0/3392663244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab402cc40 con 0x7ffabc0d2320 2024-04-01T00:14:08.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.872+0000 7ffaba7fc640 1 -- 172.21.15.64:0/3392663244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4034440 con 0x7ffabc0d2320 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 -- 172.21.15.64:0/3392663244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0d2320 msgr2=0x7ffabc06caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 --2- 172.21.15.64:0/3392663244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0d2320 0x7ffabc06caa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0d07d0 tx=0x7ffab402be90 comp rx=0 tx=0).stop 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 -- 172.21.15.64:0/3392663244 shutdown_connections 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 --2- 172.21.15.64:0/3392663244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0d2320 0x7ffabc06caa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 -- 172.21.15.64:0/3392663244 >> 172.21.15.64:0/3392663244 conn(0x7ffabc09a980 msgr2=0x7ffabc09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 -- 172.21.15.64:0/3392663244 shutdown_connections 2024-04-01T00:14:08.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.873+0000 7ffac1d93640 1 -- 172.21.15.64:0/3392663244 wait complete. 2024-04-01T00:14:08.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.874+0000 7ffac1d93640 1 Processor -- start 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.874+0000 7ffac1d93640 1 -- start start 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffabc0d2320 0x7ffabc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffabc15e1c0 0x7ffabc0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc0b0b80 con 0x7ffabc0cc340 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc0b0a00 con 0x7ffabc15e1c0 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffac1d93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffabc0b0d00 con 0x7ffabc0d2320 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52760/0 (socket says 172.21.15.64:52760) 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffabb7fe640 1 -- 172.21.15.64:0/532118106 learned_addr learned my addr 172.21.15.64:0/532118106 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffabbfff640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffabc15e1c0 0x7ffabc0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.875+0000 7ffabaffd640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffabc0d2320 0x7ffabc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 -- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffabc0d2320 msgr2=0x7ffabc152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffabc0d2320 0x7ffabc152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 -- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffabc15e1c0 msgr2=0x7ffabc0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffabc15e1c0 0x7ffabc0cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 -- 172.21.15.64:0/532118106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1605f0 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffabb7fe640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c3c0 tx=0x7ffaac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0052d0 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaac005490 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc153060 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac007260 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.876+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc160b40 con 0x7ffabc0cc340 2024-04-01T00:14:08.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.877+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc0681a0 con 0x7ffabc0cc340 2024-04-01T00:14:08.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.878+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ffaac007400 con 0x7ffabc0cc340 2024-04-01T00:14:08.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.878+0000 7ffaa3fff640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 0x7ffaa806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:08.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.878+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffaac089c70 con 0x7ffabc0cc340 2024-04-01T00:14:08.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.878+0000 7ffabaffd640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 0x7ffaa806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:08.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.879+0000 7ffabaffd640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 0x7ffaa806de50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffabc15eeb0 tx=0x7ffab0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:08.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:08.883+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ffaac058230 con 0x7ffabc0cc340 2024-04-01T00:14:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.104+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffabc160d10 con 0x7ffabc0cc340 2024-04-01T00:14:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.104+0000 7ffaa3fff640 1 -- 172.21.15.64:0/532118106 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7ffaac05c2d0 con 0x7ffabc0cc340 2024-04-01T00:14:09.107 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-01T00:14:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 msgr2=0x7ffaa806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 0x7ffaa806de50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffabc15eeb0 tx=0x7ffab0009040 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 msgr2=0x7ffabc0c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c3c0 tx=0x7ffaac00c880 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 shutdown_connections 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaa806b990 0x7ffaa806de50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffabc15e1c0 0x7ffabc0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffabc0d2320 0x7ffabc152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 --2- 172.21.15.64:0/532118106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffabc0cc340 0x7ffabc0c5820 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.106+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 >> 172.21.15.64:0/532118106 conn(0x7ffabc09a980 msgr2=0x7ffabc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.107+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 shutdown_connections 2024-04-01T00:14:09.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:09.107+0000 7ffac1d93640 1 -- 172.21.15.64:0/532118106 wait complete. 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-01T00:14:09.119 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-01T00:14:09.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.120 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-01T00:14:09.135 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42735sec preferred_lft 42735sec 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ee:18:8f:17:21:90 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::d0dc:4fff:fe82:69f3/64 scope link 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ee:18:8f:17:21:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec18:8fff:fe17:2190/64 scope link tentative 2024-04-01T00:14:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.138 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-01T00:14:09.201 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-01T00:14:09.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.203 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-01T00:14:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-01T00:14:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-01T00:14:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-01T00:14:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-01T00:14:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-01T00:14:09.259 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:09.259 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:09.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-01T00:14:09.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-01T00:14:09.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-01T00:14:09.260 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:09.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:09.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.393 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-01T00:14:09.408 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-01T00:14:09.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.408 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-01T00:14:09.463 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-01T00:14:09.463 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:09.464 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:09.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T00:14:09.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-01T00:14:09.464 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:09.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-01T00:14:09.616 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:09.616 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:09.616 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:09.616 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.616 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 86:f7:e7:b5:bf:8e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::84f7:e7ff:feb5:bf8e/64 scope link tentative 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:09.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-01T00:14:09.683 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:09.683 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:09.683 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:09.683 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.683 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 86:f7:e7:b5:bf:8e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::84f7:e7ff:feb5:bf8e/64 scope link tentative 2024-04-01T00:14:09.684 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:09.685 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-01T00:14:09.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-01T00:14:09.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-01T00:14:09.686 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:09.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:09.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:09.851 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:09.851 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-01T00:14:09.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-04-01T00:14:09.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-04-01T00:14:09.851 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-01T00:14:09.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:09.940 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-01T00:14:09.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:09.940 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:09.954 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-01T00:14:09.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:09.955 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:10.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-01T00:14:10.078 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:10.133 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-01T00:14:10.133 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-01T00:14:10.134 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-01T00:14:10.135 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T00:14:10.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.136 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-01T00:14:10.189 INFO:teuthology.orchestra.run.smithi100.stdout:41 2024-04-01T00:14:10.189 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-01T00:14:10.190 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-01T00:14:10.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-01T00:14:10.259 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-01T00:14:10.314 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T00:14:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T00:14:10.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.317 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-01T00:14:10.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:2024-04-01T00:14:10.318+0000 7fc05f185480 -1 init, newargv = 0x55da1a7b4e30 newargc=15 2024-04-01T00:14:10.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[44055]: starting ceph client 2024-04-01T00:14:10.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[44055]: starting fuse 2024-04-01T00:14:10.334 INFO:teuthology.orchestra.run.smithi100.stdout:41 2024-04-01T00:14:10.334 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-01T00:14:10.335 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-01T00:14:10.335 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.3")) 2024-04-01T00:14:10.336 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-01T00:14:10.455 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.3.44055.asok 2024-04-01T00:14:10.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-01T00:14:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:10.465 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44055.asok 2024-04-01T00:14:10.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44055.asok status 2024-04-01T00:14:10.603 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "384f31f3d3aee7f8ba3f60846b8844c049165583", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1977-g384f31f3 (384f31f3d3aee7f8ba3f60846b8844c049165583) squid (dev)", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "3", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "44055", 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4699, 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-01T00:14:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4699 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2870839111 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2870839111 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-01T00:14:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4699 192.168.0.2:0/2870839111", 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.2:0/2870839111", 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 7, 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 20, 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-01T00:14:10.606 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-01T00:14:10.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.612 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-04-01T00:14:10.612 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:10.631 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-04-01T00:14:10.631 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:10.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:10.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.704 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-04-01T00:14:10.704 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:10.760 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-04-01T00:14:10.760 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:10.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:10.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.833 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-01T00:14:10.833 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:10.849 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-01T00:14:10.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:10.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:10.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.917 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-01T00:14:10.917 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:10.973 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-01T00:14:10.973 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:10.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:10.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:11.042 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-01T00:14:11.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:11.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.167+0000 7f557149d640 1 Processor -- start 2024-04-01T00:14:11.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.167+0000 7f557149d640 1 -- start start 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f557149d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0723d0 0x7f556c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f557149d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f556c0d0650 0x7f556c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f557149d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f556c15fde0 con 0x7f556c070eb0 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f557149d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f556c1600c0 con 0x7f556c0d0650 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f557149d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f556c1603a0 con 0x7f556c0723d0 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f556c0d0650 0x7f556c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f556c0d0650 0x7f556c0cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52772/0 (socket says 172.21.15.64:52772) 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556a7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f556c070eb0 0x7f556c0cff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45976/0 (socket says 172.21.15.64:45976) 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 -- 172.21.15.64:0/887212974 learned_addr learned my addr 172.21.15.64:0/887212974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556affd640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0723d0 0x7f556c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 -- 172.21.15.64:0/887212974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0723d0 msgr2=0x7f556c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0723d0 0x7f556c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 -- 172.21.15.64:0/887212974 >> v1:172.21.15.64:6789/0 conn(0x7f556c070eb0 legacy=0x7f556c0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f5569ffb640 1 -- 172.21.15.64:0/887212974 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079922020 0 0) 0x7f556c15fde0 con 0x7f556c070eb0 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.168+0000 7f556b7fe640 1 -- 172.21.15.64:0/887212974 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c160680 con 0x7f556c0d0650 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.169+0000 7f556b7fe640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c0d0650 0x7f556c0cc620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5564009e50 tx=0x7f556402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb627a8d0f306128 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.169+0000 7f5569ffb640 1 -- 172.21.15.64:0/887212974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556402ca70 con 0x7f556c0d0650 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.169+0000 7f5569ffb640 1 -- 172.21.15.64:0/887212974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556402cc10 con 0x7f556c0d0650 2024-04-01T00:14:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.169+0000 7f5569ffb640 1 -- 172.21.15.64:0/887212974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55640343f0 con 0x7f556c0d0650 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 -- 172.21.15.64:0/887212974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c0d0650 msgr2=0x7f556c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c0d0650 0x7f556c0cc620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5564009e50 tx=0x7f556402bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 -- 172.21.15.64:0/887212974 shutdown_connections 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c0d0650 0x7f556c0cc620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 --2- 172.21.15.64:0/887212974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0723d0 0x7f556c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 -- 172.21.15.64:0/887212974 >> 172.21.15.64:0/887212974 conn(0x7f556c06b4e0 msgr2=0x7f556c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 -- 172.21.15.64:0/887212974 shutdown_connections 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.170+0000 7f557149d640 1 -- 172.21.15.64:0/887212974 wait complete. 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.171+0000 7f557149d640 1 Processor -- start 2024-04-01T00:14:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.171+0000 7f557149d640 1 -- start start 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f556c0723d0 0x7f556c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0d0650 0x7f556c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0718a0 con 0x7f556c070eb0 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f556c071720 con 0x7f556c0723d0 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f557149d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f556c0727b0 con 0x7f556c0d0650 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52776/0 (socket says 172.21.15.64:52776) 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 -- 172.21.15.64:0/685644962 learned_addr learned my addr 172.21.15.64:0/685644962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556b7fe640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0d0650 0x7f556c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556a7fc640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f556c0723d0 0x7f556c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 -- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0d0650 msgr2=0x7f556c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0d0650 0x7f556c15ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 -- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f556c0723d0 msgr2=0x7f556c152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f556c0723d0 0x7f556c152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 -- 172.21.15.64:0/685644962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c0afa50 con 0x7f556c070eb0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.172+0000 7f556affd640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9b0 tx=0x7f555c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.173+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c017070 con 0x7f556c070eb0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.173+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c00c9a0 con 0x7f556c070eb0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.173+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c0afd30 con 0x7f556c070eb0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.173+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd90 con 0x7f556c070eb0 2024-04-01T00:14:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.173+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c0b02f0 con 0x7f556c070eb0 2024-04-01T00:14:11.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.174+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c0681a0 con 0x7f556c070eb0 2024-04-01T00:14:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.174+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f555c01a070 con 0x7f556c070eb0 2024-04-01T00:14:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.175+0000 7f55537fe640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 0x7f555806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.175+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f555c08aaf0 con 0x7f556c070eb0 2024-04-01T00:14:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.175+0000 7f556a7fc640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 0x7f555806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.175+0000 7f556a7fc640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 0x7f555806de50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f556c151140 tx=0x7f5560007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.179+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f555c059140 con 0x7f556c070eb0 2024-04-01T00:14:11.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.362+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f556c0ce250 con 0x7f556c070eb0 2024-04-01T00:14:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.363+0000 7f55537fe640 1 -- 172.21.15.64:0/685644962 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f555c05d1e0 con 0x7f556c070eb0 2024-04-01T00:14:11.365 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:11.366 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 msgr2=0x7f555806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 0x7f555806de50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f556c151140 tx=0x7f5560007de0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 msgr2=0x7f556c0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9b0 tx=0x7f555c00ee70 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 shutdown_connections 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f555806b990 0x7f555806de50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f556c0d0650 0x7f556c15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f556c0723d0 0x7f556c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 --2- 172.21.15.64:0/685644962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f556c070eb0 0x7f556c0b0af0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.365+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 >> 172.21.15.64:0/685644962 conn(0x7f556c06b4e0 msgr2=0x7f556c15ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.366+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 shutdown_connections 2024-04-01T00:14:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.366+0000 7f557149d640 1 -- 172.21.15.64:0/685644962 wait complete. 2024-04-01T00:14:11.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:11.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.482+0000 7f6aca823640 1 Processor -- start 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.482+0000 7f6aca823640 1 -- start start 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6aca823640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 0x7f6ac40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6aca823640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ac40c5fb0 0x7f6ac4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6aca823640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac4144ae0 con 0x7f6ac4162290 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6aca823640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac40cb200 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6aca823640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac4162af0 con 0x7f6ac4144700 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ac40c5fb0 0x7f6ac4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ac40c5fb0 0x7f6ac4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52782/0 (socket says 172.21.15.64:52782) 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 0x7f6ac40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 -- 172.21.15.64:0/1790393807 learned_addr learned my addr 172.21.15.64:0/1790393807 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 -- 172.21.15.64:0/1790393807 >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 msgr2=0x7f6ac40c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 --2- 172.21.15.64:0/1790393807 >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 0x7f6ac40c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 -- 172.21.15.64:0/1790393807 >> v1:172.21.15.100:6789/0 conn(0x7f6ac4162290 legacy=0x7f6ac4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac37fe640 1 -- 172.21.15.64:0/1790393807 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40ceab0 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac2ffd640 1 -- 172.21.15.64:0/1790393807 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379159631 0 0) 0x7f6ac4144ae0 con 0x7f6ac4162290 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.483+0000 7f6ac3fff640 1 --2- 172.21.15.64:0/1790393807 >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 0x7f6ac40c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.484+0000 7f6ac37fe640 1 --2- 172.21.15.64:0/1790393807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac4161d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ac406db30 tx=0x7f6ab402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51324eb0b6d06a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.484+0000 7f6ac2ffd640 1 -- 172.21.15.64:0/1790393807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4035020 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.484+0000 7f6ac2ffd640 1 -- 172.21.15.64:0/1790393807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab4031870 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.484+0000 7f6ac2ffd640 1 -- 172.21.15.64:0/1790393807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4031c40 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 -- 172.21.15.64:0/1790393807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac40c5fb0 msgr2=0x7f6ac4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 --2- 172.21.15.64:0/1790393807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac4161d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ac406db30 tx=0x7f6ab402b910 comp rx=0 tx=0).stop 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 -- 172.21.15.64:0/1790393807 shutdown_connections 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 --2- 172.21.15.64:0/1790393807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac4161d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 --2- 172.21.15.64:0/1790393807 >> v2:172.21.15.100:3300/0 conn(0x7f6ac4144700 0x7f6ac40c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 -- 172.21.15.64:0/1790393807 >> 172.21.15.64:0/1790393807 conn(0x7f6ac40d0bb0 msgr2=0x7f6ac40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 -- 172.21.15.64:0/1790393807 shutdown_connections 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.485+0000 7f6aca823640 1 -- 172.21.15.64:0/1790393807 wait complete. 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.486+0000 7f6aca823640 1 Processor -- start 2024-04-01T00:14:11.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.486+0000 7f6aca823640 1 -- start start 2024-04-01T00:14:11.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 0x7f6ac415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40d0880 con 0x7f6ac4162290 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40cf070 con 0x7f6ac40c5fb0 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac40d0a00 con 0x7f6ac4144700 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52792/0 (socket says 172.21.15.64:52792) 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac40cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:59412/0 (socket says 172.21.15.64:59412) 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 -- 172.21.15.64:0/3944808858 learned_addr learned my addr 172.21.15.64:0/3944808858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac37fe640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 0x7f6ac415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 -- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 msgr2=0x7f6ac415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 0x7f6ac415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 -- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 msgr2=0x7f6ac40cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac40cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 -- 172.21.15.64:0/3944808858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40c9540 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac37fe640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 0x7f6ac415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac8dc0640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c790 tx=0x7f6abc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc003da0 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc005070 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac40c9820 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc005460 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.487+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac40ca0c0 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.488+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac40c9c10 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.489+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6abc005600 con 0x7f6ac4162290 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.489+0000 7f6ac0ff9640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 0x7f6aa406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.489+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6abc08a660 con 0x7f6ac4162290 2024-04-01T00:14:11.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.489+0000 7f6ac3fff640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 0x7f6aa406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.490+0000 7f6ac3fff640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 0x7f6aa406de50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009560 tx=0x7f6aac009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.493+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6abc058c20 con 0x7f6ac4162290 2024-04-01T00:14:11.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.673+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ac41626a0 con 0x7f6ac4162290 2024-04-01T00:14:11.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.674+0000 7f6ac0ff9640 1 -- 172.21.15.64:0/3944808858 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6ac41626a0 con 0x7f6ac4162290 2024-04-01T00:14:11.676 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 msgr2=0x7f6aa406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 0x7f6aa406de50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009560 tx=0x7f6aac009190 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 msgr2=0x7f6ac40cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c790 tx=0x7f6abc00cc50 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 shutdown_connections 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6aa406b990 0x7f6aa406de50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac4162290 0x7f6ac40cbb70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ac4144700 0x7f6ac415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 --2- 172.21.15.64:0/3944808858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac40c5fb0 0x7f6ac40cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 >> 172.21.15.64:0/3944808858 conn(0x7f6ac40d0bb0 msgr2=0x7f6ac406fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 shutdown_connections 2024-04-01T00:14:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.676+0000 7f6aca823640 1 -- 172.21.15.64:0/3944808858 wait complete. 2024-04-01T00:14:11.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:11.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 Processor -- start 2024-04-01T00:14:11.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 -- start start 2024-04-01T00:14:11.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09d806cee0 0x7f09d815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09d815de20 0x7f09d8167d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d80b1ce0 con 0x7f09d806cb00 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f09d80b1310 con 0x7f09d815de20 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.793+0000 7f09ddd02640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f09d80b1fa0 con 0x7f09d806cee0 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09d815de20 0x7f09d8167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09d815de20 0x7f09d8167d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52794/0 (socket says 172.21.15.64:52794) 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09d806cee0 0x7f09d815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 -- 172.21.15.64:0/3310688507 learned_addr learned my addr 172.21.15.64:0/3310688507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 -- 172.21.15.64:0/3310688507 >> v2:172.21.15.100:3300/0 conn(0x7f09d806cee0 msgr2=0x7f09d815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 --2- 172.21.15.64:0/3310688507 >> v2:172.21.15.100:3300/0 conn(0x7f09d806cee0 0x7f09d815d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 -- 172.21.15.64:0/3310688507 >> v1:172.21.15.100:6789/0 conn(0x7f09d806cb00 legacy=0x7f09d80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.794+0000 7f09d7fff640 1 -- 172.21.15.64:0/3310688507 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d80b2280 con 0x7f09d815de20 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09d7fff640 1 --2- 172.21.15.64:0/3310688507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d815de20 0x7f09d8167d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f09d806e970 tx=0x7f09d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7b001bec9e2406 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09d67fc640 1 -- 172.21.15.64:0/3310688507 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d002ca80 con 0x7f09d815de20 2024-04-01T00:14:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09d67fc640 1 -- 172.21.15.64:0/3310688507 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d002cc20 con 0x7f09d815de20 2024-04-01T00:14:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09d67fc640 1 -- 172.21.15.64:0/3310688507 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d00343f0 con 0x7f09d815de20 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 -- 172.21.15.64:0/3310688507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d815de20 msgr2=0x7f09d8167d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3310688507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d815de20 0x7f09d8167d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f09d806e970 tx=0x7f09d002bee0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 -- 172.21.15.64:0/3310688507 shutdown_connections 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3310688507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d815de20 0x7f09d8167d90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3310688507 >> v2:172.21.15.100:3300/0 conn(0x7f09d806cee0 0x7f09d815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 -- 172.21.15.64:0/3310688507 >> 172.21.15.64:0/3310688507 conn(0x7f09d809a980 msgr2=0x7f09d809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 -- 172.21.15.64:0/3310688507 shutdown_connections 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.795+0000 7f09ddd02640 1 -- 172.21.15.64:0/3310688507 wait complete. 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.796+0000 7f09ddd02640 1 Processor -- start 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 -- start start 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09d806cee0 0x7f09d80d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d815de20 0x7f09d8158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09d815e270 con 0x7f09d806cb00 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80b1960 con 0x7f09d815de20 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09ddd02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f09d815e3f0 con 0x7f09d806cee0 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52802/0 (socket says 172.21.15.64:52802) 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.797+0000 7f09d77fe640 1 -- 172.21.15.64:0/3897898646 learned_addr learned my addr 172.21.15.64:0/3897898646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d6ffd640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09d806cee0 0x7f09d80d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d7fff640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d815de20 0x7f09d8158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 -- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09d806cee0 msgr2=0x7f09d80d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09d806cee0 0x7f09d80d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 -- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d815de20 msgr2=0x7f09d8158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d815de20 0x7f09d8158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 -- 172.21.15.64:0/3897898646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d80b5cb0 con 0x7f09d806cb00 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09d77fe640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c3c0 tx=0x7f09c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80052d0 con 0x7f09d806cb00 2024-04-01T00:14:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c8005490 con 0x7f09d806cb00 2024-04-01T00:14:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8007260 con 0x7f09d806cb00 2024-04-01T00:14:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d80b5f90 con 0x7f09d806cb00 2024-04-01T00:14:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.798+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d80b6550 con 0x7f09d806cb00 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.799+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d80681a0 con 0x7f09d806cb00 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.800+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f09c8007400 con 0x7f09d806cb00 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.800+0000 7f09cffff640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 0x7f09b806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.800+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f09c8089be0 con 0x7f09d806cb00 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.800+0000 7f09d6ffd640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 0x7f09b806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.801+0000 7f09d6ffd640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 0x7f09b806de50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09d806f4b0 tx=0x7f09c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:11.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.805+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f09c8058230 con 0x7f09d806cb00 2024-04-01T00:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.987+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09d80679f0 con 0x7f09d806cb00 2024-04-01T00:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.987+0000 7f09cffff640 1 -- 172.21.15.64:0/3897898646 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f09c805c2d0 con 0x7f09d806cb00 2024-04-01T00:14:11.989 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:11.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 msgr2=0x7f09b806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 0x7f09b806de50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09d806f4b0 tx=0x7f09c0009190 comp rx=0 tx=0).stop 2024-04-01T00:14:11.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 msgr2=0x7f09d815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c3c0 tx=0x7f09c800c880 comp rx=0 tx=0).stop 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 shutdown_connections 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f09b806b990 0x7f09b806de50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09d815de20 0x7f09d8158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09d806cee0 0x7f09d80d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 --2- 172.21.15.64:0/3897898646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d806cb00 0x7f09d815e660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 >> 172.21.15.64:0/3897898646 conn(0x7f09d809a980 msgr2=0x7f09d80af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 shutdown_connections 2024-04-01T00:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:11.989+0000 7f09ddd02640 1 -- 172.21.15.64:0/3897898646 wait complete. 2024-04-01T00:14:12.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:12.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.105+0000 7f1f317c9640 1 Processor -- start 2024-04-01T00:14:12.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.105+0000 7f1f317c9640 1 -- start start 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f317c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f317c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0d15e0 0x7f1f2c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f317c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2c0cee70 con 0x7f1f2c15e3e0 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f317c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f1f2c0cf150 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f317c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c0cf430 con 0x7f1f2c0d15e0 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52818/0 (socket says 172.21.15.64:52818) 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 -- 172.21.15.64:0/1328864592 learned_addr learned my addr 172.21.15.64:0/1328864592 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f2affd640 1 --1- 172.21.15.64:0/1328864592 >> v1:172.21.15.64:6789/0 conn(0x7f1f2c15e3e0 0x7f1f2c0cfc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45984/0 (socket says 172.21.15.64:45984) 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f2b7fe640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0d15e0 0x7f1f2c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 -- 172.21.15.64:0/1328864592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0d15e0 msgr2=0x7f1f2c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0d15e0 0x7f1f2c0d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 -- 172.21.15.64:0/1328864592 >> v1:172.21.15.64:6789/0 conn(0x7f1f2c15e3e0 legacy=0x7f1f2c0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f23fff640 1 -- 172.21.15.64:0/1328864592 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0cf710 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.106+0000 7f1f2a7fc640 1 -- 172.21.15.64:0/1328864592 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314239808 0 0) 0x7f1f2c0cee70 con 0x7f1f2c15e3e0 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.107+0000 7f1f23fff640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aaf9791f17b1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.107+0000 7f1f2a7fc640 1 -- 172.21.15.64:0/1328864592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14034070 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.107+0000 7f1f2a7fc640 1 -- 172.21.15.64:0/1328864592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14030840 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.107+0000 7f1f2a7fc640 1 -- 172.21.15.64:0/1328864592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14030c10 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 -- 172.21.15.64:0/1328864592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 msgr2=0x7f1f2c0d4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b980 comp rx=0 tx=0).stop 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 -- 172.21.15.64:0/1328864592 shutdown_connections 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0d15e0 0x7f1f2c0d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 --2- 172.21.15.64:0/1328864592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c0d4720 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 -- 172.21.15.64:0/1328864592 >> 172.21.15.64:0/1328864592 conn(0x7f1f2c09d210 msgr2=0x7f1f2c0d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 -- 172.21.15.64:0/1328864592 shutdown_connections 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.108+0000 7f1f317c9640 1 -- 172.21.15.64:0/1328864592 wait complete. 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.109+0000 7f1f317c9640 1 Processor -- start 2024-04-01T00:14:12.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.109+0000 7f1f317c9640 1 -- start start 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 0x7f1f2c0b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c15e3e0 0x7f1f2c1f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c0d4de0 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c0d4c60 con 0x7f1f2c0d15e0 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c0d2810 con 0x7f1f2c15e3e0 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52824/0 (socket says 172.21.15.64:52824) 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 -- 172.21.15.64:0/3608890795 learned_addr learned my addr 172.21.15.64:0/3608890795 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f23fff640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c15e3e0 0x7f1f2c1f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2affd640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 0x7f1f2c0b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 -- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c15e3e0 msgr2=0x7f1f2c1f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c15e3e0 0x7f1f2c1f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 -- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 msgr2=0x7f1f2c0b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 0x7f1f2c0b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 -- 172.21.15.64:0/3608890795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0b9000 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2affd640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 0x7f1f2c0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f2b7fe640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009e30 tx=0x7f1f2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24018020 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f24014480 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c0b92e0 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24014870 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.110+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c0b98a0 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.111+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2c0681a0 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.112+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1f24005d10 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.112+0000 7f1f237fe640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 0x7f1f0c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.112+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2408b5c0 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.112+0000 7f1f2affd640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 0x7f1f0c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.113+0000 7f1f2affd640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 0x7f1f0c06de50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c000f60 tx=0x7f1f1c007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.118+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1f24022030 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.296+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f2c09d720 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.296+0000 7f1f237fe640 1 -- 172.21.15.64:0/3608890795 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1f24010040 con 0x7f1f2c15e7c0 2024-04-01T00:14:12.298 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 msgr2=0x7f1f0c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 0x7f1f0c06de50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c000f60 tx=0x7f1f1c007c10 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 msgr2=0x7f1f2c1f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009e30 tx=0x7f1f2400c490 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 shutdown_connections 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1f0c06b990 0x7f1f0c06de50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c15e7c0 0x7f1f2c1f76c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c15e3e0 0x7f1f2c1f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 --2- 172.21.15.64:0/3608890795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0d15e0 0x7f1f2c0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 >> 172.21.15.64:0/3608890795 conn(0x7f1f2c09d210 msgr2=0x7f1f2c161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 shutdown_connections 2024-04-01T00:14:12.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.298+0000 7f1f317c9640 1 -- 172.21.15.64:0/3608890795 wait complete. 2024-04-01T00:14:12.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.411+0000 7f313ce23640 1 Processor -- start 2024-04-01T00:14:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.411+0000 7f313ce23640 1 -- start start 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.412+0000 7f313ce23640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31380b4350 0x7f313806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.412+0000 7f313ce23640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3138147420 0x7f313806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.412+0000 7f313ce23640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31380b5d20 con 0x7f3138147970 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.412+0000 7f313ce23640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3138161e10 con 0x7f3138147420 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.412+0000 7f313ce23640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f31381620b0 con 0x7f31380b4350 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3138147420 0x7f313806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3138147420 0x7f313806b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52838/0 (socket says 172.21.15.64:52838) 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 -- 172.21.15.64:0/571375965 learned_addr learned my addr 172.21.15.64:0/571375965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3136d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3138147970 0x7f3138071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46000/0 (socket says 172.21.15.64:46000) 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f313659c640 1 --2- 172.21.15.64:0/571375965 >> v2:172.21.15.100:3300/0 conn(0x7f31380b4350 0x7f313806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 -- 172.21.15.64:0/571375965 >> v2:172.21.15.100:3300/0 conn(0x7f31380b4350 msgr2=0x7f313806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 --2- 172.21.15.64:0/571375965 >> v2:172.21.15.100:3300/0 conn(0x7f31380b4350 0x7f313806b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 -- 172.21.15.64:0/571375965 >> v1:172.21.15.64:6789/0 conn(0x7f3138147970 legacy=0x7f3138071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.413+0000 7f3135d9b640 1 -- 172.21.15.64:0/571375965 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3138162390 con 0x7f3138147420 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f3135d9b640 1 --2- 172.21.15.64:0/571375965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147420 0x7f313806b600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3138073950 tx=0x7f312c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb03d26303c74d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313559a640 1 -- 172.21.15.64:0/571375965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c034070 con 0x7f3138147420 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313559a640 1 -- 172.21.15.64:0/571375965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c030860 con 0x7f3138147420 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313559a640 1 -- 172.21.15.64:0/571375965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c030c30 con 0x7f3138147420 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 -- 172.21.15.64:0/571375965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147420 msgr2=0x7f313806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 --2- 172.21.15.64:0/571375965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147420 0x7f313806b600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3138073950 tx=0x7f312c02b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 -- 172.21.15.64:0/571375965 shutdown_connections 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 --2- 172.21.15.64:0/571375965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147420 0x7f313806b600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 --2- 172.21.15.64:0/571375965 >> v2:172.21.15.100:3300/0 conn(0x7f31380b4350 0x7f313806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 -- 172.21.15.64:0/571375965 >> 172.21.15.64:0/571375965 conn(0x7f31380b2760 msgr2=0x7f31380b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 -- 172.21.15.64:0/571375965 shutdown_connections 2024-04-01T00:14:12.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.414+0000 7f313ce23640 1 -- 172.21.15.64:0/571375965 wait complete. 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 Processor -- start 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 -- start start 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31380b4350 0x7f313813e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138147420 0x7f31380ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f313806ee70 con 0x7f3138147970 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3138162950 con 0x7f3138147420 2024-04-01T00:14:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f313ce23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f313806eff0 con 0x7f31380b4350 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.416+0000 7f3136d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52850/0 (socket says 172.21.15.64:52850) 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 -- 172.21.15.64:0/2899739490 learned_addr learned my addr 172.21.15.64:0/2899739490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f313659c640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31380b4350 0x7f313813e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3135d9b640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138147420 0x7f31380ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 -- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31380b4350 msgr2=0x7f313813e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31380b4350 0x7f313813e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 -- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138147420 msgr2=0x7f31380ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138147420 0x7f31380ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 -- 172.21.15.64:0/2899739490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3138140030 con 0x7f3138147970 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3136d9d640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f312800c780 tx=0x7f312800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128017070 con 0x7f3138147970 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312800a640 con 0x7f3138147970 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312800aa30 con 0x7f3138147970 2024-04-01T00:14:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313813d700 con 0x7f3138147970 2024-04-01T00:14:12.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.417+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313813dcc0 con 0x7f3138147970 2024-04-01T00:14:12.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.418+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31380681a0 con 0x7f3138147970 2024-04-01T00:14:12.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.419+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f312801a070 con 0x7f3138147970 2024-04-01T00:14:12.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.419+0000 7f3126ffd640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 0x7f311806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.419+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3128089e50 con 0x7f3138147970 2024-04-01T00:14:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.419+0000 7f313659c640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 0x7f311806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.420+0000 7f313659c640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 0x7f311806de50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3130009aa0 tx=0x7f3130009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.424+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3128058390 con 0x7f3138147970 2024-04-01T00:14:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.605+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f313806bf70 con 0x7f3138147970 2024-04-01T00:14:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.606+0000 7f3126ffd640 1 -- 172.21.15.64:0/2899739490 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f312805c430 con 0x7f3138147970 2024-04-01T00:14:12.608 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:12.608 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 msgr2=0x7f311806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 0x7f311806de50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3130009aa0 tx=0x7f3130009190 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 msgr2=0x7f31380cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f312800c780 tx=0x7f312800cc40 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 shutdown_connections 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f311806b990 0x7f311806de50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138147970 0x7f31380cf2b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138147420 0x7f31380ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 --2- 172.21.15.64:0/2899739490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31380b4350 0x7f313813e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 >> 172.21.15.64:0/2899739490 conn(0x7f31380b2760 msgr2=0x7f31380d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 shutdown_connections 2024-04-01T00:14:12.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.609+0000 7f313ce23640 1 -- 172.21.15.64:0/2899739490 wait complete. 2024-04-01T00:14:12.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:12.621 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.724+0000 7f572971e640 1 Processor -- start 2024-04-01T00:14:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.724+0000 7f572971e640 1 -- start start 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f572415e3e0 0x7f57240cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 0x7f57240d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57240d15e0 0x7f57240d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f57240cee70 con 0x7f572415e3e0 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f57240cf150 con 0x7f57240d15e0 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f572971e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f57240cf430 con 0x7f572415e7c0 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f572415e3e0 0x7f57240cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f572415e3e0 0x7f57240cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52864/0 (socket says 172.21.15.64:52864) 2024-04-01T00:14:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 -- 172.21.15.64:0/1268597977 learned_addr learned my addr 172.21.15.64:0/1268597977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f57227fc640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 0x7f57240d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f57237fe640 1 --2- 172.21.15.64:0/1268597977 >> v2:172.21.15.100:3300/0 conn(0x7f57240d15e0 0x7f57240d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 -- 172.21.15.64:0/1268597977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 msgr2=0x7f57240d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 0x7f57240d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 -- 172.21.15.64:0/1268597977 >> v2:172.21.15.100:3300/0 conn(0x7f57240d15e0 msgr2=0x7f57240d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 --2- 172.21.15.64:0/1268597977 >> v2:172.21.15.100:3300/0 conn(0x7f57240d15e0 0x7f57240d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f5722ffd640 1 -- 172.21.15.64:0/1268597977 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57240cf710 con 0x7f572415e3e0 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.725+0000 7f57227fc640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 0x7f57240d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.726+0000 7f5722ffd640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e3e0 0x7f57240cfc30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57240d1fc0 tx=0x7f571c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3d04e84126c667 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.726+0000 7f5721ffb640 1 -- 172.21.15.64:0/1268597977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c035020 con 0x7f572415e3e0 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.726+0000 7f5721ffb640 1 -- 172.21.15.64:0/1268597977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c031870 con 0x7f572415e3e0 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.726+0000 7f5721ffb640 1 -- 172.21.15.64:0/1268597977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c031c40 con 0x7f572415e3e0 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 -- 172.21.15.64:0/1268597977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e3e0 msgr2=0x7f57240cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e3e0 0x7f57240cfc30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57240d1fc0 tx=0x7f571c02b910 comp rx=0 tx=0).stop 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 -- 172.21.15.64:0/1268597977 shutdown_connections 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 --2- 172.21.15.64:0/1268597977 >> v2:172.21.15.100:3300/0 conn(0x7f57240d15e0 0x7f57240d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f572415e7c0 0x7f57240d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 --2- 172.21.15.64:0/1268597977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e3e0 0x7f57240cfc30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 -- 172.21.15.64:0/1268597977 >> 172.21.15.64:0/1268597977 conn(0x7f572409d210 msgr2=0x7f57240d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 -- 172.21.15.64:0/1268597977 shutdown_connections 2024-04-01T00:14:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.727+0000 7f572971e640 1 -- 172.21.15.64:0/1268597977 wait complete. 2024-04-01T00:14:12.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.728+0000 7f572971e640 1 Processor -- start 2024-04-01T00:14:12.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.728+0000 7f572971e640 1 -- start start 2024-04-01T00:14:12.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57240d15e0 0x7f5724152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f572415e3e0 0x7f57241f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f57240cec50 con 0x7f572415e7c0 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57240cead0 con 0x7f572415e3e0 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f572971e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f57240d4c60 con 0x7f57240d15e0 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52876/0 (socket says 172.21.15.64:52876) 2024-04-01T00:14:12.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 -- 172.21.15.64:0/3368495272 learned_addr learned my addr 172.21.15.64:0/3368495272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f5722ffd640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57240d15e0 0x7f5724152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57227fc640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f572415e3e0 0x7f57241f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 -- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57240d15e0 msgr2=0x7f5724152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57240d15e0 0x7f5724152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 -- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f572415e3e0 msgr2=0x7f57241f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f572415e3e0 0x7f57241f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.729+0000 7f57237fe640 1 -- 172.21.15.64:0/3368495272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241f5450 con 0x7f572415e7c0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f57237fe640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f571400eed0 tx=0x7f571400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714018020 con 0x7f572415e7c0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714014400 con 0x7f572415e7c0 2024-04-01T00:14:12.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57140147f0 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241f48d0 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.730+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241f4e90 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.731+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57240681a0 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.731+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f5714005d10 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.731+0000 7f571b7fe640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 0x7f570406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.732+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f571408b140 con 0x7f572415e7c0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.732+0000 7f5722ffd640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 0x7f570406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:12.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.732+0000 7f5722ffd640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 0x7f570406de50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:12.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.736+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f5714022030 con 0x7f572415e7c0 2024-04-01T00:14:12.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.916+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f572409d720 con 0x7f572415e7c0 2024-04-01T00:14:12.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.917+0000 7f571b7fe640 1 -- 172.21.15.64:0/3368495272 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f5714059700 con 0x7f572415e7c0 2024-04-01T00:14:12.919 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 msgr2=0x7f570406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 0x7f570406de50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c03c040 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 msgr2=0x7f572415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f571400eed0 tx=0x7f571400b480 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 shutdown_connections 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f570406b990 0x7f570406de50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f572415e7c0 0x7f572415c110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f572415e3e0 0x7f57241f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 --2- 172.21.15.64:0/3368495272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57240d15e0 0x7f5724152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 >> 172.21.15.64:0/3368495272 conn(0x7f572409d210 msgr2=0x7f57240b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 shutdown_connections 2024-04-01T00:14:12.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:12.919+0000 7f572971e640 1 -- 172.21.15.64:0/3368495272 wait complete. 2024-04-01T00:14:12.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:13.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.037+0000 7f0059255640 1 Processor -- start 2024-04-01T00:14:13.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.037+0000 7f0059255640 1 -- start start 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0059255640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0059255640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f00540cab60 0x7f0054070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0059255640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00540cb020 con 0x7f0054144720 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0059255640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0054144b00 con 0x7f00540cab60 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0059255640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f00540cb300 con 0x7f005406ed00 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f00540cab60 0x7f0054070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:42136/0 (socket says 172.21.15.64:42136) 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 -- 172.21.15.64:0/2466292052 learned_addr learned my addr 172.21.15.64:0/2466292052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f0052d9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0054144720 0x7f005406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:41994/0 (socket says 172.21.15.64:41994) 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 -- 172.21.15.64:0/2466292052 >> v2:172.21.15.100:3300/0 conn(0x7f00540cab60 msgr2=0x7f0054070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 --2- 172.21.15.64:0/2466292052 >> v2:172.21.15.100:3300/0 conn(0x7f00540cab60 0x7f0054070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 -- 172.21.15.64:0/2466292052 >> v1:172.21.15.100:6789/0 conn(0x7f0054144720 legacy=0x7f005406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.038+0000 7f005259c640 1 -- 172.21.15.64:0/2466292052 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00540cb5e0 con 0x7f005406ed00 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.039+0000 7f005259c640 1 --2- 172.21.15.64:0/2466292052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f003c00bb10 tx=0x7f003c02e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f16ce53a82f0f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.039+0000 7f0051d9b640 1 -- 172.21.15.64:0/2466292052 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c03b070 con 0x7f005406ed00 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.039+0000 7f0051d9b640 1 -- 172.21.15.64:0/2466292052 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c036440 con 0x7f005406ed00 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.039+0000 7f0051d9b640 1 -- 172.21.15.64:0/2466292052 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c036810 con 0x7f005406ed00 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 -- 172.21.15.64:0/2466292052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 msgr2=0x7f005406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 --2- 172.21.15.64:0/2466292052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f003c00bb10 tx=0x7f003c02e150 comp rx=0 tx=0).stop 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 -- 172.21.15.64:0/2466292052 shutdown_connections 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 --2- 172.21.15.64:0/2466292052 >> v2:172.21.15.100:3300/0 conn(0x7f00540cab60 0x7f0054070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 --2- 172.21.15.64:0/2466292052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406ed00 0x7f005406ce70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 -- 172.21.15.64:0/2466292052 >> 172.21.15.64:0/2466292052 conn(0x7f00540cfa90 msgr2=0x7f00540cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 -- 172.21.15.64:0/2466292052 shutdown_connections 2024-04-01T00:14:13.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.040+0000 7f0059255640 1 -- 172.21.15.64:0/2466292052 wait complete. 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.041+0000 7f0059255640 1 Processor -- start 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.041+0000 7f0059255640 1 -- start start 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 0x7f00540ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0054144720 0x7f00540c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00540cd7f0 con 0x7f005406ed00 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00540cbba0 con 0x7f0054144720 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0059255640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f00540cd970 con 0x7f00540cab60 2024-04-01T00:14:13.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0052d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0052d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52888/0 (socket says 172.21.15.64:52888) 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0052d9d640 1 -- 172.21.15.64:0/621207730 learned_addr learned my addr 172.21.15.64:0/621207730 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 0x7f00540ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005359e640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0054144720 0x7f00540c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 -- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0054144720 msgr2=0x7f00540c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0054144720 0x7f00540c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 -- 172.21.15.64:0/621207730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 msgr2=0x7f00540ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f005259c640 1 -- 172.21.15.64:0/621207730 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00540af160 con 0x7f00540cab60 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.042+0000 7f0052d9d640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f005259c640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 0x7f00540ae6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f003c035000 tx=0x7f003c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c03b070 con 0x7f00540cab60 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c03d050 con 0x7f00540cab60 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00540aee80 con 0x7f00540cab60 2024-04-01T00:14:13.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c034b30 con 0x7f00540cab60 2024-04-01T00:14:13.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.043+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0054142790 con 0x7f00540cab60 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.044+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00540681a0 con 0x7f00540cab60 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.044+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f003c048060 con 0x7f00540cab60 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.045+0000 7f004b7fe640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 0x7f003406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.045+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f003c0ac1c0 con 0x7f00540cab60 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.045+0000 7f0052d9d640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 0x7f003406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.045+0000 7f0052d9d640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 0x7f003406de50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f004c007fd0 tx=0x7f004c0016a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.049+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f003c07a810 con 0x7f00540cab60 2024-04-01T00:14:13.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.232+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00540cf230 con 0x7f00540cab60 2024-04-01T00:14:13.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.232+0000 7f004b7fe640 1 -- 172.21.15.64:0/621207730 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f003c07e8b0 con 0x7f00540cab60 2024-04-01T00:14:13.233 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 msgr2=0x7f003406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 0x7f003406de50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f004c007fd0 tx=0x7f004c0016a0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 msgr2=0x7f00540ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 0x7f00540ae6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f003c035000 tx=0x7f003c0046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 shutdown_connections 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f003406b990 0x7f003406de50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0054144720 0x7f00540c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cab60 0x7f00540ae6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 --2- 172.21.15.64:0/621207730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f005406ed00 0x7f00540ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 >> 172.21.15.64:0/621207730 conn(0x7f00540cfa90 msgr2=0x7f00540b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 shutdown_connections 2024-04-01T00:14:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.234+0000 7f0059255640 1 -- 172.21.15.64:0/621207730 wait complete. 2024-04-01T00:14:13.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.350+0000 7fe1cbc3b640 1 Processor -- start 2024-04-01T00:14:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.350+0000 7fe1cbc3b640 1 -- start start 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1cbc3b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1c415e3e0 0x7fe1c40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1cbc3b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1c40d15e0 0x7fe1c40d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1cbc3b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c40cee70 con 0x7fe1c415e7c0 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1cbc3b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c40cf150 con 0x7fe1c415e3e0 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1cbc3b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c40cf430 con 0x7fe1c40d15e0 2024-04-01T00:14:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1c415e3e0 0x7fe1c40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1c415e3e0 0x7fe1c40cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52902/0 (socket says 172.21.15.64:52902) 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 -- 172.21.15.64:0/269256128 learned_addr learned my addr 172.21.15.64:0/269256128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1ca1d8640 1 --2- 172.21.15.64:0/269256128 >> v2:172.21.15.100:3300/0 conn(0x7fe1c40d15e0 0x7fe1c40d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 -- 172.21.15.64:0/269256128 >> v2:172.21.15.100:3300/0 conn(0x7fe1c40d15e0 msgr2=0x7fe1c40d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 --2- 172.21.15.64:0/269256128 >> v2:172.21.15.100:3300/0 conn(0x7fe1c40d15e0 0x7fe1c40d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 -- 172.21.15.64:0/269256128 >> v1:172.21.15.100:6789/0 conn(0x7fe1c415e7c0 legacy=0x7fe1c40d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.351+0000 7fe1c99d7640 1 -- 172.21.15.64:0/269256128 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c40cf710 con 0x7fe1c415e3e0 2024-04-01T00:14:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.352+0000 7fe1c99d7640 1 --2- 172.21.15.64:0/269256128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c40cfc30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40c7a30 tx=0x7fe1bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b793101f51c8e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.352+0000 7fe1c89d5640 1 -- 172.21.15.64:0/269256128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc034070 con 0x7fe1c415e3e0 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.352+0000 7fe1c89d5640 1 -- 172.21.15.64:0/269256128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc030860 con 0x7fe1c415e3e0 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.352+0000 7fe1c89d5640 1 -- 172.21.15.64:0/269256128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc030c30 con 0x7fe1c415e3e0 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/269256128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 msgr2=0x7fe1c40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/269256128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c40cfc30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40c7a30 tx=0x7fe1bc02b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/269256128 shutdown_connections 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/269256128 >> v2:172.21.15.100:3300/0 conn(0x7fe1c40d15e0 0x7fe1c40d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/269256128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c40cfc30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/269256128 >> 172.21.15.64:0/269256128 conn(0x7fe1c409d210 msgr2=0x7fe1c40d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/269256128 shutdown_connections 2024-04-01T00:14:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.353+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/269256128 wait complete. 2024-04-01T00:14:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.354+0000 7fe1cbc3b640 1 Processor -- start 2024-04-01T00:14:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.354+0000 7fe1cbc3b640 1 -- start start 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c40d15e0 0x7fe1c40ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c415e7c0 0x7fe1c415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c40cec50 con 0x7fe1c415e3e0 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c40cead0 con 0x7fe1c40d15e0 2024-04-01T00:14:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c40d2810 con 0x7fe1c415e7c0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52906/0 (socket says 172.21.15.64:52906) 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 -- 172.21.15.64:0/1754054281 learned_addr learned my addr 172.21.15.64:0/1754054281 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c99d7640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c40d15e0 0x7fe1c40ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1ca1d8640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c415e7c0 0x7fe1c415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 -- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c415e7c0 msgr2=0x7fe1c415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c415e7c0 0x7fe1c415c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 -- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c40d15e0 msgr2=0x7fe1c40ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c40d15e0 0x7fe1c40ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 -- 172.21.15.64:0/1754054281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c40c88b0 con 0x7fe1c415e3e0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c91d6640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c3c0 tx=0x7fe1b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80052d0 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b8005490 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8007260 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.355+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c40c8b90 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.356+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c40c9150 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.356+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c40b7740 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.357+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe1b8007400 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.357+0000 7fe1c27fc640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 0x7fe1b006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.357+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b8089c30 con 0x7fe1c415e3e0 2024-04-01T00:14:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.357+0000 7fe1c99d7640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 0x7fe1b006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.358+0000 7fe1c99d7640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 0x7fe1b006de50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc036000 tx=0x7fe1bc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.362+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe1b8058280 con 0x7fe1c415e3e0 2024-04-01T00:14:13.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.542+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1c409d720 con 0x7fe1c415e3e0 2024-04-01T00:14:13.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.542+0000 7fe1c27fc640 1 -- 172.21.15.64:0/1754054281 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe1b805c320 con 0x7fe1c415e3e0 2024-04-01T00:14:13.545 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 msgr2=0x7fe1b006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 0x7fe1b006de50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc036000 tx=0x7fe1bc008030 comp rx=0 tx=0).stop 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 msgr2=0x7fe1c415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c3c0 tx=0x7fe1b800c880 comp rx=0 tx=0).stop 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 shutdown_connections 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe1b006b990 0x7fe1b006de50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c415e7c0 0x7fe1c415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c415e3e0 0x7fe1c415be20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 --2- 172.21.15.64:0/1754054281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c40d15e0 0x7fe1c40ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 >> 172.21.15.64:0/1754054281 conn(0x7fe1c409d210 msgr2=0x7fe1c415d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 shutdown_connections 2024-04-01T00:14:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.544+0000 7fe1cbc3b640 1 -- 172.21.15.64:0/1754054281 wait complete. 2024-04-01T00:14:13.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:13.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.657+0000 7f3de1aed640 1 Processor -- start 2024-04-01T00:14:13.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.657+0000 7f3de1aed640 1 -- start start 2024-04-01T00:14:13.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3de1aed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3de1aed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3de1aed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc0d1ff0 con 0x7f3ddc0ceba0 2024-04-01T00:14:13.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3de1aed640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3ddc0b5830 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3de1aed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cf000 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3ddb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3ddb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33110/0 (socket says 172.21.15.64:33110) 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3038035562 learned_addr learned my addr 172.21.15.64:0/3038035562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.658+0000 7f3ddbfff640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3038035562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 msgr2=0x7f3ddc0b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddb7fe640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc0b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3038035562 >> v1:172.21.15.100:6789/0 conn(0x7f3ddc0ceba0 legacy=0x7f3ddc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3038035562 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0cf280 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3dda7fc640 1 -- 172.21.15.64:0/3038035562 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795219669 0 0) 0x7f3dd0001110 con 0x7f3ddc0ceba0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddbfff640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc0b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.659+0000 7f3ddb7fe640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009870 tx=0x7f3dcc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3de08517c72d156 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3dda7fc640 1 -- 172.21.15.64:0/3038035562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc034070 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3dda7fc640 1 -- 172.21.15.64:0/3038035562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc030840 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3dda7fc640 1 -- 172.21.15.64:0/3038035562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc030c10 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 -- 172.21.15.64:0/3038035562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 msgr2=0x7f3ddc0d4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009870 tx=0x7f3dcc02b980 comp rx=0 tx=0).stop 2024-04-01T00:14:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 -- 172.21.15.64:0/3038035562 shutdown_connections 2024-04-01T00:14:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc0b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3038035562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc0d4a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 -- 172.21.15.64:0/3038035562 >> 172.21.15.64:0/3038035562 conn(0x7f3ddc0d3ee0 msgr2=0x7f3ddc0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 -- 172.21.15.64:0/3038035562 shutdown_connections 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.660+0000 7f3de1aed640 1 -- 172.21.15.64:0/3038035562 wait complete. 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.661+0000 7f3de1aed640 1 Processor -- start 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 -- start start 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ddc0ceba0 0x7f3ddc160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cfb20 con 0x7f3ddc0d08f0 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cf840 con 0x7f3ddc0ceba0 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3de1aed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cfe00 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3ddb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33116/0 (socket says 172.21.15.64:33116) 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3ddbfff640 1 -- 172.21.15.64:0/3002531926 learned_addr learned my addr 172.21.15.64:0/3002531926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.662+0000 7f3ddaffd640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ddc0ceba0 0x7f3ddc160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ddc0ceba0 msgr2=0x7f3ddc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ddc0ceba0 0x7f3ddc160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3002531926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 msgr2=0x7f3ddc161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 -- 172.21.15.64:0/3002531926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc14a720 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddbfff640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3ddb7fe640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc149c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc033000 tx=0x7f3dcc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc048070 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc030640 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc034040 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc14a1a0 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.663+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc0c8900 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.664+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc0681a0 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.665+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3dcc04b070 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.665+0000 7f3dc3fff640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 0x7f3dc806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.665+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3dcc0ac590 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.665+0000 7f3ddaffd640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 0x7f3dc806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.666+0000 7f3ddaffd640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 0x7f3dc806de50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0d1bb0 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.670+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3ddc14a720 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.851+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ddc0d22a0 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.852+0000 7f3dc3fff640 1 -- 172.21.15.64:0/3002531926 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f3dcc07ad00 con 0x7f3ddc0b6dc0 2024-04-01T00:14:13.853 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:13.853 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 msgr2=0x7f3dc806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 0x7f3dc806de50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0d1bb0 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 msgr2=0x7f3ddc149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc149c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc033000 tx=0x7f3dcc004730 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 shutdown_connections 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3dc806b990 0x7f3dc806de50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ddc0d08f0 0x7f3ddc161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ddc0ceba0 0x7f3ddc160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 --2- 172.21.15.64:0/3002531926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ddc0b6dc0 0x7f3ddc149c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 >> 172.21.15.64:0/3002531926 conn(0x7f3ddc0d3ee0 msgr2=0x7f3ddc06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 shutdown_connections 2024-04-01T00:14:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.854+0000 7f3de1aed640 1 -- 172.21.15.64:0/3002531926 wait complete. 2024-04-01T00:14:13.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:13.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:13.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.970+0000 7f417317a640 1 Processor -- start 2024-04-01T00:14:13.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.970+0000 7f417317a640 1 -- start start 2024-04-01T00:14:13.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f417317a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 0x7f416c166b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f417317a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c0c65f0 con 0x7f416c0cf4c0 2024-04-01T00:14:13.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f417317a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c0cf9f0 con 0x7f416c1459a0 2024-04-01T00:14:13.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f417317a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0cfcd0 con 0x7f416c06b7d0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f416c0cf4c0 0x7f416c06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49734/0 (socket says 172.21.15.64:49734) 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416bfff640 1 -- 172.21.15.64:0/722839432 learned_addr learned my addr 172.21.15.64:0/722839432 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f4171717640 1 --2- 172.21.15.64:0/722839432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 0x7f416c166b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319727800 0 0) 0x7f416c0c65f0 con 0x7f416c0cf4c0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4154003520 con 0x7f416c0cf4c0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752837493 0 0) 0x7f416c0cf9f0 con 0x7f416c1459a0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f416c0c65f0 con 0x7f416c1459a0 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f4171717640 1 -- 172.21.15.64:0/722839432 >> v1:172.21.15.100:6789/0 conn(0x7f416c1459a0 legacy=0x7f416c0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f4171717640 1 -- 172.21.15.64:0/722839432 >> v1:172.21.15.64:6789/0 conn(0x7f416c0cf4c0 legacy=0x7f416c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.971+0000 7f4171717640 1 -- 172.21.15.64:0/722839432 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c0cffb0 con 0x7f416c06b7d0 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.972+0000 7f4171717640 1 --2- 172.21.15.64:0/722839432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 0x7f416c166b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097d0 tx=0x7f415c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc81e1bd3412abf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.972+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c035020 con 0x7f416c06b7d0 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.972+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415c031890 con 0x7f416c06b7d0 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.972+0000 7f416b7fe640 1 -- 172.21.15.64:0/722839432 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c031c40 con 0x7f416c06b7d0 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 -- 172.21.15.64:0/722839432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 msgr2=0x7f416c166b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 --2- 172.21.15.64:0/722839432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 0x7f416c166b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097d0 tx=0x7f415c02b930 comp rx=0 tx=0).stop 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 -- 172.21.15.64:0/722839432 shutdown_connections 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 --2- 172.21.15.64:0/722839432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c06b7d0 0x7f416c166b90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 -- 172.21.15.64:0/722839432 >> 172.21.15.64:0/722839432 conn(0x7f416c0d0bb0 msgr2=0x7f416c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 -- 172.21.15.64:0/722839432 shutdown_connections 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.973+0000 7f417317a640 1 -- 172.21.15.64:0/722839432 wait complete. 2024-04-01T00:14:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.974+0000 7f417317a640 1 Processor -- start 2024-04-01T00:14:13.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.974+0000 7f417317a640 1 -- start start 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0cf4c0 0x7f416c15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c1459a0 0x7f416c1f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f416c06ed30 con 0x7f416c06b7d0 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0d0570 con 0x7f416c0cf4c0 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f417317a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f416c06eeb0 con 0x7f416c1459a0 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f4170f16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f4170f16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33126/0 (socket says 172.21.15.64:33126) 2024-04-01T00:14:13.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f4170f16640 1 -- 172.21.15.64:0/238772087 learned_addr learned my addr 172.21.15.64:0/238772087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f416bfff640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0cf4c0 0x7f416c15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.975+0000 7f4171717640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c1459a0 0x7f416c1f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 -- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c1459a0 msgr2=0x7f416c1f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c1459a0 0x7f416c1f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 -- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0cf4c0 msgr2=0x7f416c15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0cf4c0 0x7f416c15ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 -- 172.21.15.64:0/238772087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1f2990 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f4170f16640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f416400c7d0 tx=0x7f416400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164003b70 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4164003d30 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1f23d0 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400aa30 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.976+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c0b3a20 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.977+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c0681a0 con 0x7f416c06b7d0 2024-04-01T00:14:13.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.977+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f416400abd0 con 0x7f416c06b7d0 2024-04-01T00:14:13.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.978+0000 7f41697fa640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 0x7f415806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:13.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.978+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f416408a520 con 0x7f416c06b7d0 2024-04-01T00:14:13.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.978+0000 7f416bfff640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 0x7f415806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:13.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.979+0000 7f416bfff640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 0x7f415806de50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f416c0c9840 tx=0x7f4160007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:13.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:13.982+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4164058b70 con 0x7f416c06b7d0 2024-04-01T00:14:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.162+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f416c072b50 con 0x7f416c06b7d0 2024-04-01T00:14:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.163+0000 7f41697fa640 1 -- 172.21.15.64:0/238772087 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f416405cc10 con 0x7f416c06b7d0 2024-04-01T00:14:14.164 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.164+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 msgr2=0x7f415806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.164+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 0x7f415806de50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f416c0c9840 tx=0x7f4160007d30 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.164+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 msgr2=0x7f416c0c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.164+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f416400c7d0 tx=0x7f416400cc90 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 shutdown_connections 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f415806b990 0x7f415806de50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c1459a0 0x7f416c1f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0cf4c0 0x7f416c15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 --2- 172.21.15.64:0/238772087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416c06b7d0 0x7f416c0c7c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 >> 172.21.15.64:0/238772087 conn(0x7f416c0d0bb0 msgr2=0x7f416c0c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 shutdown_connections 2024-04-01T00:14:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.165+0000 7f417317a640 1 -- 172.21.15.64:0/238772087 wait complete. 2024-04-01T00:14:14.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:14.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.280+0000 7fad8fd93640 1 Processor -- start 2024-04-01T00:14:14.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.280+0000 7fad8fd93640 1 -- start start 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8fd93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad880cab60 0x7fad8806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8fd93640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad880cb020 con 0x7fad88144720 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8fd93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad880cb300 con 0x7fad8806ed00 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8fd93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fad88144b00 con 0x7fad880cab60 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8db2f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fad88144720 0x7fad8806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49740/0 (socket says 172.21.15.64:49740) 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8db2f640 1 -- 172.21.15.64:0/1833262327 learned_addr learned my addr 172.21.15.64:0/1833262327 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8e330640 1 --2- 172.21.15.64:0/1833262327 >> v2:172.21.15.100:3300/0 conn(0x7fad880cab60 0x7fad8806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152029978 0 0) 0x7fad880cb020 con 0x7fad88144720 2024-04-01T00:14:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad74003520 con 0x7fad88144720 2024-04-01T00:14:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8e330640 1 -- 172.21.15.64:0/1833262327 >> v1:172.21.15.100:6789/0 conn(0x7fad8806ed00 legacy=0x7fad88070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 133879220 0 0) 0x7fad880cb300 con 0x7fad8806ed00 2024-04-01T00:14:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8e330640 1 -- 172.21.15.64:0/1833262327 >> v1:172.21.15.64:6789/0 conn(0x7fad88144720 legacy=0x7fad8806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.281+0000 7fad8e330640 1 -- 172.21.15.64:0/1833262327 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad880cb5e0 con 0x7fad880cab60 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.282+0000 7fad8e330640 1 --2- 172.21.15.64:0/1833262327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad880cab60 0x7fad8806ce70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad80009d90 tx=0x7fad8002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270795c829d74f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.282+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80039070 con 0x7fad880cab60 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.282+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8002cd00 con 0x7fad880cab60 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.282+0000 7fad8cb2d640 1 -- 172.21.15.64:0/1833262327 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80034640 con 0x7fad880cab60 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 -- 172.21.15.64:0/1833262327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad880cab60 msgr2=0x7fad8806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 --2- 172.21.15.64:0/1833262327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad880cab60 0x7fad8806ce70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad80009d90 tx=0x7fad8002bf50 comp rx=0 tx=0).stop 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 -- 172.21.15.64:0/1833262327 shutdown_connections 2024-04-01T00:14:14.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 --2- 172.21.15.64:0/1833262327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad880cab60 0x7fad8806ce70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 -- 172.21.15.64:0/1833262327 >> 172.21.15.64:0/1833262327 conn(0x7fad880cfa90 msgr2=0x7fad880cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 -- 172.21.15.64:0/1833262327 shutdown_connections 2024-04-01T00:14:14.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.283+0000 7fad8fd93640 1 -- 172.21.15.64:0/1833262327 wait complete. 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.284+0000 7fad8fd93640 1 Processor -- start 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.284+0000 7fad8fd93640 1 -- start start 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8806ed00 0x7fad8815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad88144720 0x7fad881f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad880cd7f0 con 0x7fad880cab60 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad880cbba0 con 0x7fad8806ed00 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fad880cd970 con 0x7fad88144720 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33134/0 (socket says 172.21.15.64:33134) 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 -- 172.21.15.64:0/4234009056 learned_addr learned my addr 172.21.15.64:0/4234009056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8db2f640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8806ed00 0x7fad8815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8e330640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad88144720 0x7fad881f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 -- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad88144720 msgr2=0x7fad881f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad88144720 0x7fad881f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 -- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8806ed00 msgr2=0x7fad8815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8806ed00 0x7fad8815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 -- 172.21.15.64:0/4234009056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad881f2b30 con 0x7fad880cab60 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8d32e640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad7800c4e0 tx=0x7fad7800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78017070 con 0x7fad880cab60 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad780035f0 con 0x7fad880cab60 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad881f2e10 con 0x7fad880cab60 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780039e0 con 0x7fad880cab60 2024-04-01T00:14:14.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.285+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad881f33d0 con 0x7fad880cab60 2024-04-01T00:14:14.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.287+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad880681a0 con 0x7fad880cab60 2024-04-01T00:14:14.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.287+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fad7801a070 con 0x7fad880cab60 2024-04-01T00:14:14.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.287+0000 7fad7e7fc640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 0x7fad6c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.287+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad78089b10 con 0x7fad880cab60 2024-04-01T00:14:14.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.288+0000 7fad8db2f640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 0x7fad6c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.288+0000 7fad8db2f640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 0x7fad6c06de50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad881f51a0 tx=0x7fad84000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.292+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fad780580d0 con 0x7fad880cab60 2024-04-01T00:14:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.471+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad880cf230 con 0x7fad880cab60 2024-04-01T00:14:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.472+0000 7fad7e7fc640 1 -- 172.21.15.64:0/4234009056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fad7805c170 con 0x7fad880cab60 2024-04-01T00:14:14.473 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 msgr2=0x7fad6c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 0x7fad6c06de50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad881f51a0 tx=0x7fad84000bc0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 msgr2=0x7fad881e6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad7800c4e0 tx=0x7fad7800c9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 shutdown_connections 2024-04-01T00:14:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fad6c06b990 0x7fad6c06de50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad88144720 0x7fad881f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad880cab60 0x7fad881e6f50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 --2- 172.21.15.64:0/4234009056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8806ed00 0x7fad8815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 >> 172.21.15.64:0/4234009056 conn(0x7fad880cfa90 msgr2=0x7fad880721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 shutdown_connections 2024-04-01T00:14:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.474+0000 7fad8fd93640 1 -- 172.21.15.64:0/4234009056 wait complete. 2024-04-01T00:14:14.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:14.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.590+0000 7fa1f7c07640 1 Processor -- start 2024-04-01T00:14:14.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.590+0000 7fa1f7c07640 1 -- start start 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f7c07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 0x7fa1f00ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f7c07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1f00cd810 0x7fa1f00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f7c07640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00cc350 con 0x7fa1f00d2780 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f7c07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00d0aa0 con 0x7fa1f00cd810 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f7c07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0cf0 con 0x7fa1f015e250 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f51a2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1f00d2780 0x7fa1f006c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49754/0 (socket says 172.21.15.64:49754) 2024-04-01T00:14:14.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f51a2640 1 -- 172.21.15.64:0/3277119678 learned_addr learned my addr 172.21.15.64:0/3277119678 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/3277119678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 0x7fa1f00ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f61a4640 1 --2- 172.21.15.64:0/3277119678 >> v2:172.21.15.100:3300/0 conn(0x7fa1f00cd810 0x7fa1f00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f59a3640 1 -- 172.21.15.64:0/3277119678 >> v2:172.21.15.100:3300/0 conn(0x7fa1f00cd810 msgr2=0x7fa1f00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/3277119678 >> v2:172.21.15.100:3300/0 conn(0x7fa1f00cd810 0x7fa1f00cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f49a1640 1 -- 172.21.15.64:0/3277119678 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111983595 0 0) 0x7fa1f00cc350 con 0x7fa1f00d2780 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f59a3640 1 -- 172.21.15.64:0/3277119678 >> v1:172.21.15.64:6789/0 conn(0x7fa1f00d2780 legacy=0x7fa1f006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.591+0000 7fa1f59a3640 1 -- 172.21.15.64:0/3277119678 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00d0fd0 con 0x7fa1f015e250 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.592+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/3277119678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 0x7fa1f00ce4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097a0 tx=0x7fa1ec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58afd7f51e586f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.592+0000 7fa1f49a1640 1 -- 172.21.15.64:0/3277119678 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0315a0 con 0x7fa1f015e250 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.592+0000 7fa1f49a1640 1 -- 172.21.15.64:0/3277119678 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa1f015e250 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.592+0000 7fa1f49a1640 1 -- 172.21.15.64:0/3277119678 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec031960 con 0x7fa1f015e250 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 -- 172.21.15.64:0/3277119678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 msgr2=0x7fa1f00ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/3277119678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 0x7fa1f00ce4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097a0 tx=0x7fa1ec02b840 comp rx=0 tx=0).stop 2024-04-01T00:14:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 -- 172.21.15.64:0/3277119678 shutdown_connections 2024-04-01T00:14:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/3277119678 >> v2:172.21.15.100:3300/0 conn(0x7fa1f00cd810 0x7fa1f00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/3277119678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f015e250 0x7fa1f00ce4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 -- 172.21.15.64:0/3277119678 >> 172.21.15.64:0/3277119678 conn(0x7fa1f00d1c60 msgr2=0x7fa1f00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 -- 172.21.15.64:0/3277119678 shutdown_connections 2024-04-01T00:14:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.593+0000 7fa1f7c07640 1 -- 172.21.15.64:0/3277119678 wait complete. 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.594+0000 7fa1f7c07640 1 Processor -- start 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.594+0000 7fa1f7c07640 1 -- start start 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 0x7fa1f00c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1f00d2780 0x7fa1f014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f002ce60 con 0x7fa1f015e250 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f002cce0 con 0x7fa1f00d2780 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f7c07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00b2f20 con 0x7fa1f00cd810 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f59a3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 0x7fa1f00c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33146/0 (socket says 172.21.15.64:33146) 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 -- 172.21.15.64:0/1408025355 learned_addr learned my addr 172.21.15.64:0/1408025355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f51a2640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1f00d2780 0x7fa1f014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 -- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 msgr2=0x7fa1f00c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 0x7fa1f00c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 -- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1f00d2780 msgr2=0x7fa1f014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1f00d2780 0x7fa1f014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 -- 172.21.15.64:0/1408025355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00c8090 con 0x7fa1f015e250 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 0x7fa1f00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:14.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1f61a4640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800efc0 tx=0x7fa1e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.595+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8018020 con 0x7fa1f015e250 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.596+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8014480 con 0x7fa1f015e250 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.596+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00c8370 con 0x7fa1f015e250 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.596+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8014870 con 0x7fa1f015e250 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.596+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f014ab70 con 0x7fa1f015e250 2024-04-01T00:14:14.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.596+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00681a0 con 0x7fa1f015e250 2024-04-01T00:14:14.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.597+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fa1e8005d10 con 0x7fa1f015e250 2024-04-01T00:14:14.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.597+0000 7fa1e67fc640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 0x7fa1d406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.597+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e808b700 con 0x7fa1f015e250 2024-04-01T00:14:14.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.598+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 0x7fa1d406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.598+0000 7fa1f59a3640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 0x7fa1d406de50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec036000 tx=0x7fa1ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.602+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fa1e8022030 con 0x7fa1f015e250 2024-04-01T00:14:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.786+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1f00ce090 con 0x7fa1f015e250 2024-04-01T00:14:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.786+0000 7fa1e67fc640 1 -- 172.21.15.64:0/1408025355 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa1e8010040 con 0x7fa1f015e250 2024-04-01T00:14:14.787 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 msgr2=0x7fa1d406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 0x7fa1d406de50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec036000 tx=0x7fa1ec008030 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 msgr2=0x7fa1f014d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800efc0 tx=0x7fa1e800c490 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 shutdown_connections 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa1d406b990 0x7fa1d406de50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1f015e250 0x7fa1f014d030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1f00d2780 0x7fa1f014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 --2- 172.21.15.64:0/1408025355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1f00cd810 0x7fa1f00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 >> 172.21.15.64:0/1408025355 conn(0x7fa1f00d1c60 msgr2=0x7fa1f00c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 shutdown_connections 2024-04-01T00:14:14.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.788+0000 7fa1f7c07640 1 -- 172.21.15.64:0/1408025355 wait complete. 2024-04-01T00:14:14.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:14.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.901+0000 7f15acf35640 1 Processor -- start 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.901+0000 7f15acf35640 1 -- start start 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15acf35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a81619e0 0x7f15a80d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15acf35640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a80cfae0 0x7f15a81620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15acf35640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a80b3c80 con 0x7f15a80b3820 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15acf35640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f15a80d2ba0 con 0x7f15a80cfae0 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15acf35640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15a80d2e80 con 0x7f15a81619e0 2024-04-01T00:14:14.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15a6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a80cfae0 0x7f15a81620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15a5d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f15a80b3820 0x7f15a80cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49756/0 (socket says 172.21.15.64:49756) 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15a6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a80cfae0 0x7f15a81620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33160/0 (socket says 172.21.15.64:33160) 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15a5d9b640 1 -- 172.21.15.64:0/4135809692 learned_addr learned my addr 172.21.15.64:0/4135809692 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.902+0000 7f15a659c640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a81619e0 0x7f15a80d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a6d9d640 1 -- 172.21.15.64:0/4135809692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a81619e0 msgr2=0x7f15a80d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a6d9d640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a81619e0 0x7f15a80d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a6d9d640 1 -- 172.21.15.64:0/4135809692 >> v1:172.21.15.64:6789/0 conn(0x7f15a80b3820 legacy=0x7f15a80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a6d9d640 1 -- 172.21.15.64:0/4135809692 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a80ceeb0 con 0x7f15a80cfae0 2024-04-01T00:14:14.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a559a640 1 -- 172.21.15.64:0/4135809692 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314780091 0 0) 0x7f15a80b3c80 con 0x7f15a80b3820 2024-04-01T00:14:14.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.903+0000 7f15a6d9d640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80cfae0 0x7f15a81620a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1598009d90 tx=0x7f159802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a459a0475ef91df server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15a559a640 1 -- 172.21.15.64:0/4135809692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598039070 con 0x7f15a80cfae0 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15a559a640 1 -- 172.21.15.64:0/4135809692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159802cbc0 con 0x7f15a80cfae0 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15a559a640 1 -- 172.21.15.64:0/4135809692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598034440 con 0x7f15a80cfae0 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 -- 172.21.15.64:0/4135809692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80cfae0 msgr2=0x7f15a81620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80cfae0 0x7f15a81620a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1598009d90 tx=0x7f159802be10 comp rx=0 tx=0).stop 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 -- 172.21.15.64:0/4135809692 shutdown_connections 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80cfae0 0x7f15a81620a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 --2- 172.21.15.64:0/4135809692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a81619e0 0x7f15a80d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 -- 172.21.15.64:0/4135809692 >> 172.21.15.64:0/4135809692 conn(0x7f15a809d290 msgr2=0x7f15a80d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 -- 172.21.15.64:0/4135809692 shutdown_connections 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.904+0000 7f15acf35640 1 -- 172.21.15.64:0/4135809692 wait complete. 2024-04-01T00:14:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.905+0000 7f15acf35640 1 Processor -- start 2024-04-01T00:14:14.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.905+0000 7f15acf35640 1 -- start start 2024-04-01T00:14:14.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a80cfae0 0x7f15a80b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a81619e0 0x7f15a814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80b31b0 con 0x7f15a80b3820 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80d2890 con 0x7f15a81619e0 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15acf35640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15a80ceb10 con 0x7f15a80cfae0 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15a659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15a659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33168/0 (socket says 172.21.15.64:33168) 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15a659c640 1 -- 172.21.15.64:0/2898653013 learned_addr learned my addr 172.21.15.64:0/2898653013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15a5d9b640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a80cfae0 0x7f15a80b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.906+0000 7f15a6d9d640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a81619e0 0x7f15a814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 -- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a80cfae0 msgr2=0x7f15a80b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a80cfae0 0x7f15a80b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 -- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a81619e0 msgr2=0x7f15a814f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a81619e0 0x7f15a814f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 -- 172.21.15.64:0/2898653013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a80b9e40 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15a659c640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e870 tx=0x7f15a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0017070 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a000c940 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a815f070 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cd30 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.907+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a815f630 con 0x7f15a80b3820 2024-04-01T00:14:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.908+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a80688b0 con 0x7f15a80b3820 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.908+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f15a001a070 con 0x7f15a80b3820 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.909+0000 7f1596ffd640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 0x7f158806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.909+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15a008a9f0 con 0x7f15a80b3820 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.909+0000 7f15a5d9b640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 0x7f158806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.909+0000 7f15a5d9b640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 0x7f158806de50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15a80b51e0 tx=0x7f159c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:14.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:14.913+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f15a80688b0 con 0x7f15a80b3820 2024-04-01T00:14:15.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.094+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a815fa70 con 0x7f15a80b3820 2024-04-01T00:14:15.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.095+0000 7f1596ffd640 1 -- 172.21.15.64:0/2898653013 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f15a008ce20 con 0x7f15a80b3820 2024-04-01T00:14:15.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 msgr2=0x7f158806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 0x7f158806de50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15a80b51e0 tx=0x7f159c007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 msgr2=0x7f15a80b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e870 tx=0x7f15a000ed30 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 shutdown_connections 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f158806b990 0x7f158806de50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a81619e0 0x7f15a814f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a80cfae0 0x7f15a80b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 --2- 172.21.15.64:0/2898653013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a80b3820 0x7f15a80b6bd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 >> 172.21.15.64:0/2898653013 conn(0x7f15a809d290 msgr2=0x7f15a80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 shutdown_connections 2024-04-01T00:14:15.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.097+0000 7f15acf35640 1 -- 172.21.15.64:0/2898653013 wait complete. 2024-04-01T00:14:15.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:15.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.213+0000 7f3f76a2e640 1 Processor -- start 2024-04-01T00:14:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.213+0000 7f3f76a2e640 1 -- start start 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f76a2e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f76a2e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f700cfa50 con 0x7f3f700d1e30 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f76a2e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f700cfcd0 con 0x7f3f70161b30 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f76a2e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f700b3140 con 0x7f3f70161750 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f74fcb640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3f700d1e30 0x7f3f700b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49762/0 (socket says 172.21.15.64:49762) 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f74fcb640 1 -- 172.21.15.64:0/4186233927 learned_addr learned my addr 172.21.15.64:0/4186233927 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/4186233927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.214+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579805517 0 0) 0x7f3f68001490 con 0x7f3f700d1e30 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f58003520 con 0x7f3f700d1e30 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547723096 0 0) 0x7f3f700cfcd0 con 0x7f3f70161b30 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f68001490 con 0x7f3f70161b30 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6ffff640 1 -- 172.21.15.64:0/4186233927 >> v1:172.21.15.100:6789/0 conn(0x7f3f70161b30 legacy=0x7f3f700d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6ffff640 1 -- 172.21.15.64:0/4186233927 >> v1:172.21.15.64:6789/0 conn(0x7f3f700d1e30 legacy=0x7f3f700b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6ffff640 1 -- 172.21.15.64:0/4186233927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f700cff90 con 0x7f3f70161750 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.215+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/4186233927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700d4a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f640097d0 tx=0x7f3f6402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758733862873f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64035020 con 0x7f3f70161750 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f64031800 con 0x7f3f70161750 2024-04-01T00:14:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f6effd640 1 -- 172.21.15.64:0/4186233927 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64031bd0 con 0x7f3f70161750 2024-04-01T00:14:15.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 -- 172.21.15.64:0/4186233927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 msgr2=0x7f3f700d4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/4186233927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700d4a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f640097d0 tx=0x7f3f6402b8a0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 -- 172.21.15.64:0/4186233927 shutdown_connections 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/4186233927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700d4a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 -- 172.21.15.64:0/4186233927 >> 172.21.15.64:0/4186233927 conn(0x7f3f7009d210 msgr2=0x7f3f700d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 -- 172.21.15.64:0/4186233927 shutdown_connections 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.216+0000 7f3f76a2e640 1 -- 172.21.15.64:0/4186233927 wait complete. 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.217+0000 7f3f76a2e640 1 Processor -- start 2024-04-01T00:14:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.217+0000 7f3f76a2e640 1 -- start start 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70161b30 0x7f3f701f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700d2b00 con 0x7f3f700d1e30 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700b3730 con 0x7f3f70161b30 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f76a2e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f700d2c80 con 0x7f3f70161750 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33178/0 (socket says 172.21.15.64:33178) 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f6ffff640 1 -- 172.21.15.64:0/3129695237 learned_addr learned my addr 172.21.15.64:0/3129695237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f74fcb640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70161b30 0x7f3f701f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.218+0000 7f3f6f7fe640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 -- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 msgr2=0x7f3f700b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 -- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70161b30 msgr2=0x7f3f701f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70161b30 0x7f3f701f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 -- 172.21.15.64:0/3129695237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f701f7be0 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6ffff640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f6402beb0 tx=0x7f3f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64035040 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6403ad00 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f701f7ec0 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640466c0 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.219+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f70152a00 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.220+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f700681a0 con 0x7f3f700d1e30 2024-04-01T00:14:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.220+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3f64038020 con 0x7f3f700d1e30 2024-04-01T00:14:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.221+0000 7f3f6cff9640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 0x7f3f5c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.221+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f640ac7a0 con 0x7f3f700d1e30 2024-04-01T00:14:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.221+0000 7f3f6f7fe640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 0x7f3f5c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.221+0000 7f3f6f7fe640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 0x7f3f5c06de50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007fd0 tx=0x7f3f60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.225+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3f6407adf0 con 0x7f3f700d1e30 2024-04-01T00:14:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.405+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f700d06e0 con 0x7f3f700d1e30 2024-04-01T00:14:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.406+0000 7f3f6cff9640 1 -- 172.21.15.64:0/3129695237 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3f6407ee90 con 0x7f3f700d1e30 2024-04-01T00:14:15.407 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 msgr2=0x7f3f5c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 0x7f3f5c06de50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007fd0 tx=0x7f3f60009040 comp rx=0 tx=0).stop 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 msgr2=0x7f3f70152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f6402beb0 tx=0x7f3f640046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 shutdown_connections 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3f5c06b990 0x7f3f5c06de50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70161b30 0x7f3f701f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70161750 0x7f3f700b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 --2- 172.21.15.64:0/3129695237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f700d1e30 0x7f3f70152f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 >> 172.21.15.64:0/3129695237 conn(0x7f3f7009d210 msgr2=0x7f3f7006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 shutdown_connections 2024-04-01T00:14:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.408+0000 7f3f76a2e640 1 -- 172.21.15.64:0/3129695237 wait complete. 2024-04-01T00:14:15.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 Processor -- start 2024-04-01T00:14:15.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 -- start start 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0338161a30 0x7f03380d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03380b3740 0x7f03380b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0338161e10 con 0x7f03380d0900 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f03380b3ba0 con 0x7f03380b3740 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f033e7c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f03380cef20 con 0x7f0338161a30 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.524+0000 7f03377fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03380b3740 0x7f03380b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f033cd66640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f03380d0900 0x7f03380cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49770/0 (socket says 172.21.15.64:49770) 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f03377fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03380b3740 0x7f03380b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33192/0 (socket says 172.21.15.64:33192) 2024-04-01T00:14:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f033cd66640 1 -- 172.21.15.64:0/798007924 learned_addr learned my addr 172.21.15.64:0/798007924 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f0337fff640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0338161a30 0x7f03380d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f03377fe640 1 -- 172.21.15.64:0/798007924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0338161a30 msgr2=0x7f03380d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f03377fe640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0338161a30 0x7f03380d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f03377fe640 1 -- 172.21.15.64:0/798007924 >> v1:172.21.15.64:6789/0 conn(0x7f03380d0900 legacy=0x7f03380cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f0336ffd640 1 -- 172.21.15.64:0/798007924 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285458362 0 0) 0x7f0330001490 con 0x7f03380d0900 2024-04-01T00:14:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.525+0000 7f03377fe640 1 -- 172.21.15.64:0/798007924 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03380d2880 con 0x7f03380b3740 2024-04-01T00:14:15.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f03377fe640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f03380b5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03200097a0 tx=0x7f032002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a763ce12f5500362 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f0336ffd640 1 -- 172.21.15.64:0/798007924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320035020 con 0x7f03380b3740 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f0336ffd640 1 -- 172.21.15.64:0/798007924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0320031830 con 0x7f03380b3740 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f0336ffd640 1 -- 172.21.15.64:0/798007924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320031c00 con 0x7f03380b3740 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 -- 172.21.15.64:0/798007924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 msgr2=0x7f03380b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f03380b5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03200097a0 tx=0x7f032002b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 -- 172.21.15.64:0/798007924 shutdown_connections 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f03380b5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 --2- 172.21.15.64:0/798007924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0338161a30 0x7f03380d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 -- 172.21.15.64:0/798007924 >> 172.21.15.64:0/798007924 conn(0x7f033809d250 msgr2=0x7f03380d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 -- 172.21.15.64:0/798007924 shutdown_connections 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.526+0000 7f033e7c9640 1 -- 172.21.15.64:0/798007924 wait complete. 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.527+0000 7f033e7c9640 1 Processor -- start 2024-04-01T00:14:15.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 -- start start 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03380d0900 0x7f03380c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0338161a30 0x7f03380c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f03380d2e00 con 0x7f03380b3740 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03380b5690 con 0x7f0338161a30 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f033e7c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f03380ceb40 con 0x7f03380d0900 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f0337fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f0337fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33200/0 (socket says 172.21.15.64:33200) 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.528+0000 7f0337fff640 1 -- 172.21.15.64:0/2612523478 learned_addr learned my addr 172.21.15.64:0/2612523478 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f033cd66640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0338161a30 0x7f03380c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f03377fe640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03380d0900 0x7f03380c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 -- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03380d0900 msgr2=0x7f03380c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03380d0900 0x7f03380c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 -- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0338161a30 msgr2=0x7f03380c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0338161a30 0x7f03380c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 -- 172.21.15.64:0/2612523478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033814e4f0 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0337fff640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f032800e9e0 tx=0x7f032800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328017070 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032800c9d0 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800cdc0 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03381513d0 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.529+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033814b250 con 0x7f03380b3740 2024-04-01T00:14:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.530+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03380681a0 con 0x7f03380b3740 2024-04-01T00:14:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.531+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f032801a070 con 0x7f03380b3740 2024-04-01T00:14:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.531+0000 7f0334ff9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 0x7f031806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.531+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f032808a7f0 con 0x7f03380b3740 2024-04-01T00:14:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.531+0000 7f03377fe640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 0x7f031806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.531+0000 7f03377fe640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 0x7f031806de80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0320036000 tx=0x7f032003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.535+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f033814e4f0 con 0x7f03380b3740 2024-04-01T00:14:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.716+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03380d1110 con 0x7f03380b3740 2024-04-01T00:14:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.717+0000 7f0334ff9640 1 -- 172.21.15.64:0/2612523478 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f0328058db0 con 0x7f03380b3740 2024-04-01T00:14:15.718 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.718+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 msgr2=0x7f031806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.718+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 0x7f031806de80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0320036000 tx=0x7f032003c040 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.718+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 msgr2=0x7f0338150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.718+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f032800e9e0 tx=0x7f032800eea0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 shutdown_connections 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b9c0 0x7f031806de80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0338161a30 0x7f03380c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03380d0900 0x7f03380c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 --2- 172.21.15.64:0/2612523478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03380b3740 0x7f0338150e90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 >> 172.21.15.64:0/2612523478 conn(0x7f033809d250 msgr2=0x7f033806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 shutdown_connections 2024-04-01T00:14:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.719+0000 7f033e7c9640 1 -- 172.21.15.64:0/2612523478 wait complete. 2024-04-01T00:14:15.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.834+0000 7f688e156640 1 Processor -- start 2024-04-01T00:14:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.834+0000 7f688e156640 1 -- start start 2024-04-01T00:14:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f688e156640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f68880cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f688e156640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68880cc730 con 0x7f68880cf740 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f688e156640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68880cca10 con 0x7f68880b0ba0 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f688e156640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68880b0f80 con 0x7f68880b1850 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f6887fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f68880cf740 0x7f68880cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49776/0 (socket says 172.21.15.64:49776) 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f6887fff640 1 -- 172.21.15.64:0/1781278128 learned_addr learned my addr 172.21.15.64:0/1781278128 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f6886ffd640 1 --2- 172.21.15.64:0/1781278128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f68880cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156104731 0 0) 0x7f68880cc730 con 0x7f68880cf740 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.835+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6870003520 con 0x7f68880cf740 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 29072552 0 0) 0x7f68880cca10 con 0x7f68880b0ba0 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f6886ffd640 1 -- 172.21.15.64:0/1781278128 >> v1:172.21.15.100:6789/0 conn(0x7f68880b0ba0 legacy=0x7f68880b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f6886ffd640 1 -- 172.21.15.64:0/1781278128 >> v1:172.21.15.64:6789/0 conn(0x7f68880cf740 legacy=0x7f68880cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f6886ffd640 1 -- 172.21.15.64:0/1781278128 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68880cccf0 con 0x7f68880b1850 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f6886ffd640 1 --2- 172.21.15.64:0/1781278128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f68880cd7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f687c0097d0 tx=0x7f687c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3edeb5b42988a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.836+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c035020 con 0x7f68880b1850 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c031890 con 0x7f68880b1850 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f68867fc640 1 -- 172.21.15.64:0/1781278128 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c031c60 con 0x7f68880b1850 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 -- 172.21.15.64:0/1781278128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 msgr2=0x7f68880cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 --2- 172.21.15.64:0/1781278128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f68880cd7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f687c0097d0 tx=0x7f687c02b930 comp rx=0 tx=0).stop 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 -- 172.21.15.64:0/1781278128 shutdown_connections 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 --2- 172.21.15.64:0/1781278128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f68880cd7d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 -- 172.21.15.64:0/1781278128 >> 172.21.15.64:0/1781278128 conn(0x7f6888072810 msgr2=0x7f6888072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 -- 172.21.15.64:0/1781278128 shutdown_connections 2024-04-01T00:14:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.837+0000 7f688e156640 1 -- 172.21.15.64:0/1781278128 wait complete. 2024-04-01T00:14:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.838+0000 7f688e156640 1 Processor -- start 2024-04-01T00:14:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.838+0000 7f688e156640 1 -- start start 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f6888099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68880cf740 0x7f688806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6888162e10 con 0x7f68880b0ba0 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68880cd2b0 con 0x7f68880cf740 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f688e156640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68880caa30 con 0x7f68880b1850 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33204/0 (socket says 172.21.15.64:33204) 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 -- 172.21.15.64:0/2406635382 learned_addr learned my addr 172.21.15.64:0/2406635382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f6887fff640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68880cf740 0x7f688806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f6886ffd640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f6888099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 -- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 msgr2=0x7f6888099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f6888099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 -- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68880cf740 msgr2=0x7f688806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68880cf740 0x7f688806e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.839+0000 7f68877fe640 1 -- 172.21.15.64:0/2406635382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68880c4520 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f68877fe640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f687800c4b0 tx=0x7f687800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878017070 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6878005480 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68881f5240 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878003830 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.840+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68881f5800 con 0x7f68880b0ba0 2024-04-01T00:14:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.841+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68880681a0 con 0x7f68880b0ba0 2024-04-01T00:14:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.841+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f687801a070 con 0x7f68880b0ba0 2024-04-01T00:14:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.842+0000 7f686ffff640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 0x7f687406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.842+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6878089c10 con 0x7f68880b0ba0 2024-04-01T00:14:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.842+0000 7f6886ffd640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 0x7f687406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.842+0000 7f6886ffd640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 0x7f687406de50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f687c036000 tx=0x7f687c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:15.846+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6878058130 con 0x7f68880b0ba0 2024-04-01T00:14:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.027+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6888072030 con 0x7f68880b0ba0 2024-04-01T00:14:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.027+0000 7f686ffff640 1 -- 172.21.15.64:0/2406635382 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f687805c1d0 con 0x7f68880b0ba0 2024-04-01T00:14:16.028 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 msgr2=0x7f687406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 0x7f687406de50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f687c036000 tx=0x7f687c008030 comp rx=0 tx=0).stop 2024-04-01T00:14:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 msgr2=0x7f68880c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f687800c4b0 tx=0x7f687800c970 comp rx=0 tx=0).stop 2024-04-01T00:14:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 shutdown_connections 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687406b990 0x7f687406de50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68880cf740 0x7f688806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68880b1850 0x7f6888099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 --2- 172.21.15.64:0/2406635382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68880b0ba0 0x7f68880c3e40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 >> 172.21.15.64:0/2406635382 conn(0x7f6888072810 msgr2=0x7f68880b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 shutdown_connections 2024-04-01T00:14:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.029+0000 7f688e156640 1 -- 172.21.15.64:0/2406635382 wait complete. 2024-04-01T00:14:16.078 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-01T00:14:16.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:16.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.180+0000 7fee03da1640 1 Processor -- start 2024-04-01T00:14:16.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.180+0000 7fee03da1640 1 -- start start 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee03da1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee03da1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee03da1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedfc0d14f0 con 0x7fedfc161830 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee03da1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0cfe00 con 0x7fedfc0d1060 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee03da1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc161c10 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee0233e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee0133c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee01b3d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fedfc161830 0x7fedfc0d0950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:43328/0 (socket says 172.21.15.64:43328) 2024-04-01T00:14:16.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee01b3d640 1 -- 172.21.15.64:0/3187733022 learned_addr learned my addr 172.21.15.64:0/3187733022 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee0133c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:56926/0 (socket says 172.21.15.64:56926) 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.181+0000 7fee0233e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:47100/0 (socket says 172.21.15.64:47100) 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee00b3b640 1 -- 172.21.15.64:0/3187733022 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2200771951 0 0) 0x7fedfc0d14f0 con 0x7fedfc161830 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee00b3b640 1 -- 172.21.15.64:0/3187733022 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede4003520 con 0x7fedfc161830 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0133c640 1 -- 172.21.15.64:0/3187733022 >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 msgr2=0x7fedfc0d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0133c640 1 --2- 172.21.15.64:0/3187733022 >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0133c640 1 -- 172.21.15.64:0/3187733022 >> v1:172.21.15.100:6789/0 conn(0x7fedfc161830 legacy=0x7fedfc0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0133c640 1 -- 172.21.15.64:0/3187733022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0d17a0 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0233e640 1 --2- 172.21.15.64:0/3187733022 >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:16.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.182+0000 7fee0133c640 1 --2- 172.21.15.64:0/3187733022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedf00097a0 tx=0x7fedf002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d54d1a919ce58dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee00b3b640 1 -- 172.21.15.64:0/3187733022 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0035020 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee00b3b640 1 -- 172.21.15.64:0/3187733022 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf0031830 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee00b3b640 1 -- 172.21.15.64:0/3187733022 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0031c00 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 -- 172.21.15.64:0/3187733022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 msgr2=0x7fedfc0b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 --2- 172.21.15.64:0/3187733022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedf00097a0 tx=0x7fedf002b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 -- 172.21.15.64:0/3187733022 shutdown_connections 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 --2- 172.21.15.64:0/3187733022 >> v2:172.21.15.100:3300/0 conn(0x7fedfc0d1060 0x7fedfc0d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 --2- 172.21.15.64:0/3187733022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0b4da0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 -- 172.21.15.64:0/3187733022 >> 172.21.15.64:0/3187733022 conn(0x7fedfc0d3fa0 msgr2=0x7fedfc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 -- 172.21.15.64:0/3187733022 shutdown_connections 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.183+0000 7fee03da1640 1 -- 172.21.15.64:0/3187733022 wait complete. 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.184+0000 7fee03da1640 1 Processor -- start 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.184+0000 7fee03da1640 1 -- start start 2024-04-01T00:14:16.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc161830 0x7fedfc14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0d20b0 con 0x7fedfc0d1060 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0cfa90 con 0x7fedfc161830 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee03da1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0d2230 con 0x7fedfc0b7fb0 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee0133c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee0133c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33206/0 (socket says 172.21.15.64:33206) 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee0133c640 1 -- 172.21.15.64:0/4065704042 learned_addr learned my addr 172.21.15.64:0/4065704042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee01b3d640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.185+0000 7fee0233e640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc161830 0x7fedfc14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 -- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 msgr2=0x7fedfc0a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 -- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc161830 msgr2=0x7fedfc14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc161830 0x7fedfc14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 -- 172.21.15.64:0/4065704042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0c89c0 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee0133c640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedf00098c0 tx=0x7fedf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0048070 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf003a630 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc0c8ca0 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf003aa20 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.186+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc0c9260 con 0x7fedfc0d1060 2024-04-01T00:14:16.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.187+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc0681a0 con 0x7fedfc0d1060 2024-04-01T00:14:16.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.187+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fedf004b070 con 0x7fedfc0d1060 2024-04-01T00:14:16.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.188+0000 7fedfa7fc640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 0x7fede806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.188+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fedf00abe60 con 0x7fedfc0d1060 2024-04-01T00:14:16.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.188+0000 7fee01b3d640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 0x7fede806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.188+0000 7fee01b3d640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 0x7fede806de50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005880 tx=0x7fedf4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.192+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fedf007a430 con 0x7fedfc0d1060 2024-04-01T00:14:16.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.372+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedfc0c7670 con 0x7fedfc0d1060 2024-04-01T00:14:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.373+0000 7fedfa7fc640 1 -- 172.21.15.64:0/4065704042 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fedf007e4d0 con 0x7fedfc0d1060 2024-04-01T00:14:16.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:16.375 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:16.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 msgr2=0x7fede806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 0x7fede806de50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005880 tx=0x7fedf4009040 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 msgr2=0x7fedfc14dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedf00098c0 tx=0x7fedf00046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 shutdown_connections 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fede806b990 0x7fede806de50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc161830 0x7fedfc14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc0d1060 0x7fedfc14dfd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 --2- 172.21.15.64:0/4065704042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedfc0b7fb0 0x7fedfc0a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 >> 172.21.15.64:0/4065704042 conn(0x7fedfc0d3fa0 msgr2=0x7fedfc1606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 shutdown_connections 2024-04-01T00:14:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.375+0000 7fee03da1640 1 -- 172.21.15.64:0/4065704042 wait complete. 2024-04-01T00:14:16.387 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:16.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:16.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:14:16.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:14:16.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.493+0000 7f07f5999640 1 Processor -- start 2024-04-01T00:14:16.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.493+0000 7f07f5999640 1 -- start start 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07f5999640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07f5999640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07f5999640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f00cb580 con 0x7f07f00d07a0 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07f5999640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f07f015fa40 con 0x7f07f006b600 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07f5999640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07f00720b0 con 0x7f07f0071c30 2024-04-01T00:14:16.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07ee7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07ef7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07ee7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:56942/0 (socket says 172.21.15.64:56942) 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07ef7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:47126/0 (socket says 172.21.15.64:47126) 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07ee7fc640 1 -- 172.21.15.64:0/3413214526 learned_addr learned my addr 172.21.15.64:0/3413214526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.494+0000 7f07eeffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f07f00d07a0 0x7f07f00ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:43330/0 (socket says 172.21.15.64:43330) 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ee7fc640 1 -- 172.21.15.64:0/3413214526 >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 msgr2=0x7f07f015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ee7fc640 1 --2- 172.21.15.64:0/3413214526 >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ee7fc640 1 -- 172.21.15.64:0/3413214526 >> v1:172.21.15.100:6789/0 conn(0x7f07f00d07a0 legacy=0x7f07f00ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ee7fc640 1 -- 172.21.15.64:0/3413214526 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f015fc90 con 0x7f07f0071c30 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07edffb640 1 -- 172.21.15.64:0/3413214526 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417381695 0 0) 0x7f07e4001110 con 0x7f07f00d07a0 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ef7fe640 1 --2- 172.21.15.64:0/3413214526 >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.495+0000 7f07ee7fc640 1 --2- 172.21.15.64:0/3413214526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009870 tx=0x7f07e002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81fb7937c1bac16a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07edffb640 1 -- 172.21.15.64:0/3413214526 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0034070 con 0x7f07f0071c30 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07edffb640 1 -- 172.21.15.64:0/3413214526 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e0030840 con 0x7f07f0071c30 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07edffb640 1 -- 172.21.15.64:0/3413214526 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0030c10 con 0x7f07f0071c30 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 -- 172.21.15.64:0/3413214526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 msgr2=0x7f07f006b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 --2- 172.21.15.64:0/3413214526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009870 tx=0x7f07e002b980 comp rx=0 tx=0).stop 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 -- 172.21.15.64:0/3413214526 shutdown_connections 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 --2- 172.21.15.64:0/3413214526 >> v2:172.21.15.100:3300/0 conn(0x7f07f006b600 0x7f07f015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 --2- 172.21.15.64:0/3413214526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f0071c30 0x7f07f006b0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 -- 172.21.15.64:0/3413214526 >> 172.21.15.64:0/3413214526 conn(0x7f07f00d0c00 msgr2=0x7f07f00d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 -- 172.21.15.64:0/3413214526 shutdown_connections 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.496+0000 7f07f5999640 1 -- 172.21.15.64:0/3413214526 wait complete. 2024-04-01T00:14:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.497+0000 7f07f5999640 1 Processor -- start 2024-04-01T00:14:16.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.497+0000 7f07f5999640 1 -- start start 2024-04-01T00:14:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0071c30 0x7f07f014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f00d07a0 0x7f07f00c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0160530 con 0x7f07f006b600 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0160250 con 0x7f07f0071c30 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07f5999640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07f01608a0 con 0x7f07f00d07a0 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07eeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07eeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33218/0 (socket says 172.21.15.64:33218) 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07eeffd640 1 -- 172.21.15.64:0/728619621 learned_addr learned my addr 172.21.15.64:0/728619621 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07ef7fe640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f00d07a0 0x7f07f00c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.498+0000 7f07ee7fc640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0071c30 0x7f07f014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 -- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f00d07a0 msgr2=0x7f07f00c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f00d07a0 0x7f07f00c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 -- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0071c30 msgr2=0x7f07f014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0071c30 0x7f07f014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 -- 172.21.15.64:0/728619621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f00c4450 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07eeffd640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c780 tx=0x7f07e400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4018070 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e40072d0 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f00c4730 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40047e0 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.499+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f01f3030 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.500+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f00688b0 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.500+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f07e4004980 con 0x7f07f006b600 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.501+0000 7f07d7fff640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 0x7f07dc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.501+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07e4089a90 con 0x7f07f006b600 2024-04-01T00:14:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.501+0000 7f07ee7fc640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 0x7f07dc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.502+0000 7f07ee7fc640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 0x7f07dc06de50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f07e0036000 tx=0x7f07e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.505+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f07f00cb580 con 0x7f07f006b600 2024-04-01T00:14:16.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.685+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07f00671f0 con 0x7f07f006b600 2024-04-01T00:14:16.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.685+0000 7f07d7fff640 1 -- 172.21.15.64:0/728619621 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f07e4004c70 con 0x7f07f006b600 2024-04-01T00:14:16.686 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:16.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 msgr2=0x7f07dc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 0x7f07dc06de50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f07e0036000 tx=0x7f07e0008030 comp rx=0 tx=0).stop 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 msgr2=0x7f07f01f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c780 tx=0x7f07e400cb80 comp rx=0 tx=0).stop 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 shutdown_connections 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f07dc06b990 0x7f07dc06de50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f00d07a0 0x7f07f00c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0071c30 0x7f07f014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 --2- 172.21.15.64:0/728619621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f006b600 0x7f07f01f2af0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 >> 172.21.15.64:0/728619621 conn(0x7f07f00d0c00 msgr2=0x7f07f015bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 shutdown_connections 2024-04-01T00:14:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.687+0000 7f07f5999640 1 -- 172.21.15.64:0/728619621 wait complete. 2024-04-01T00:14:16.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:16.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.804+0000 7f967a01e640 1 Processor -- start 2024-04-01T00:14:16.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.804+0000 7f967a01e640 1 -- start start 2024-04-01T00:14:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740cd660 0x7f96740cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96740c6500 0x7f967406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96740c5540 0x7f96740c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f96740cb690 con 0x7f96740c6500 2024-04-01T00:14:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f96740c50b0 con 0x7f96740c5540 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f967a01e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96740cb3b0 con 0x7f96740cd660 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96740c6500 0x7f967406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96740c6500 0x7f967406d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33230/0 (socket says 172.21.15.64:33230) 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 -- 172.21.15.64:0/3835475321 learned_addr learned my addr 172.21.15.64:0/3835475321 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f96737fe640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740cd660 0x7f96740cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9673fff640 1 --2- 172.21.15.64:0/3835475321 >> v2:172.21.15.100:3300/0 conn(0x7f96740c5540 0x7f96740c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 -- 172.21.15.64:0/3835475321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740cd660 msgr2=0x7f96740cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740cd660 0x7f96740cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 -- 172.21.15.64:0/3835475321 >> v2:172.21.15.100:3300/0 conn(0x7f96740c5540 msgr2=0x7f96740c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 --2- 172.21.15.64:0/3835475321 >> v2:172.21.15.100:3300/0 conn(0x7f96740c5540 0x7f96740c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.805+0000 7f9672ffd640 1 -- 172.21.15.64:0/3835475321 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96740c4a60 con 0x7f96740c6500 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.806+0000 7f9672ffd640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c6500 0x7f967406d4e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96740d0b10 tx=0x7f966802dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3310bc165342f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.806+0000 7f96727fc640 1 -- 172.21.15.64:0/3835475321 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966803b070 con 0x7f96740c6500 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.806+0000 7f96727fc640 1 -- 172.21.15.64:0/3835475321 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966802ed70 con 0x7f96740c6500 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.806+0000 7f96727fc640 1 -- 172.21.15.64:0/3835475321 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668036660 con 0x7f96740c6500 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 -- 172.21.15.64:0/3835475321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c6500 msgr2=0x7f967406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c6500 0x7f967406d4e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96740d0b10 tx=0x7f966802dfc0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 -- 172.21.15.64:0/3835475321 shutdown_connections 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 --2- 172.21.15.64:0/3835475321 >> v2:172.21.15.100:3300/0 conn(0x7f96740c5540 0x7f96740c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c6500 0x7f967406d4e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 --2- 172.21.15.64:0/3835475321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740cd660 0x7f96740cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 -- 172.21.15.64:0/3835475321 >> 172.21.15.64:0/3835475321 conn(0x7f96740cfa90 msgr2=0x7f96740cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 -- 172.21.15.64:0/3835475321 shutdown_connections 2024-04-01T00:14:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.807+0000 7f967a01e640 1 -- 172.21.15.64:0/3835475321 wait complete. 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.808+0000 7f967a01e640 1 Processor -- start 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.808+0000 7f967a01e640 1 -- start start 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740c6500 0x7f96740b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 0x7f96740b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96740cc0a0 con 0x7f96740c5540 2024-04-01T00:14:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96740c5f10 con 0x7f96740cd660 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f967a01e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f967406da20 con 0x7f96740c6500 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33240/0 (socket says 172.21.15.64:33240) 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 -- 172.21.15.64:0/2971354905 learned_addr learned my addr 172.21.15.64:0/2971354905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f9673fff640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 0x7f96740b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f9672ffd640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740c6500 0x7f96740b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 -- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740c6500 msgr2=0x7f96740b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740c6500 0x7f96740b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 -- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 msgr2=0x7f96740b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 0x7f96740b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f96737fe640 1 -- 172.21.15.64:0/2971354905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967414ec00 con 0x7f96740c5540 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.809+0000 7f9673fff640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 0x7f96740b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f96737fe640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f966400e9e0 tx=0x7f966400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664017070 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966400c9d0 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96740b44d0 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966400cdc0 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.810+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96740b4d30 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.811+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96740681a0 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.811+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f966401a070 con 0x7f96740c5540 2024-04-01T00:14:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.812+0000 7f965bfff640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 0x7f966006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.812+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f966408a860 con 0x7f96740c5540 2024-04-01T00:14:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.812+0000 7f9672ffd640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 0x7f966006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.812+0000 7f9672ffd640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 0x7f966006de50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9668038000 tx=0x7f9668008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.816+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f967414ec00 con 0x7f96740c5540 2024-04-01T00:14:16.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.995+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96740cae90 con 0x7f96740c5540 2024-04-01T00:14:16.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.996+0000 7f965bfff640 1 -- 172.21.15.64:0/2971354905 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f966408a020 con 0x7f96740c5540 2024-04-01T00:14:16.997 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 msgr2=0x7f966006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 0x7f966006de50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9668038000 tx=0x7f9668008040 comp rx=0 tx=0).stop 2024-04-01T00:14:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 msgr2=0x7f96741e3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f966400e9e0 tx=0x7f966400eea0 comp rx=0 tx=0).stop 2024-04-01T00:14:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 shutdown_connections 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f966006b990 0x7f966006de50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96740cd660 0x7f96740b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96740c6500 0x7f96740b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 --2- 172.21.15.64:0/2971354905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96740c5540 0x7f96741e3d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 >> 172.21.15.64:0/2971354905 conn(0x7f96740cfa90 msgr2=0x7f967414f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 shutdown_connections 2024-04-01T00:14:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:16.998+0000 7f967a01e640 1 -- 172.21.15.64:0/2971354905 wait complete. 2024-04-01T00:14:17.010 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.114+0000 7fd557c15640 1 Processor -- start 2024-04-01T00:14:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 -- start start 2024-04-01T00:14:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd55015e250 0x7fd5500ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd5500cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5500b2f20 0x7fd55006c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500d0aa0 con 0x7fd55015e250 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500cc350 con 0x7fd5500b2f20 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd557c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5500d0cf0 con 0x7fd5500d2780 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd5559b1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd55015e250 0x7fd5500ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd5559b1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd55015e250 0x7fd5500ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33244/0 (socket says 172.21.15.64:33244) 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.115+0000 7fd5559b1640 1 -- 172.21.15.64:0/2189936160 learned_addr learned my addr 172.21.15.64:0/2189936160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5551b0640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd5500cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5561b2640 1 --2- 172.21.15.64:0/2189936160 >> v2:172.21.15.100:3300/0 conn(0x7fd5500b2f20 0x7fd55006c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5559b1640 1 -- 172.21.15.64:0/2189936160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 msgr2=0x7fd5500cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5559b1640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd5500cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5559b1640 1 -- 172.21.15.64:0/2189936160 >> v2:172.21.15.100:3300/0 conn(0x7fd5500b2f20 msgr2=0x7fd55006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5559b1640 1 --2- 172.21.15.64:0/2189936160 >> v2:172.21.15.100:3300/0 conn(0x7fd5500b2f20 0x7fd55006c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.116+0000 7fd5559b1640 1 -- 172.21.15.64:0/2189936160 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500d0fd0 con 0x7fd55015e250 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd5559b1640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55015e250 0x7fd5500ce4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd5500b7ce0 tx=0x7fd54802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21771e35a10c812 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd5549af640 1 -- 172.21.15.64:0/2189936160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54802e070 con 0x7fd55015e250 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd5549af640 1 -- 172.21.15.64:0/2189936160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd548037070 con 0x7fd55015e250 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd5549af640 1 -- 172.21.15.64:0/2189936160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548031a00 con 0x7fd55015e250 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 -- 172.21.15.64:0/2189936160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55015e250 msgr2=0x7fd5500ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55015e250 0x7fd5500ce4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd5500b7ce0 tx=0x7fd54802b880 comp rx=0 tx=0).stop 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 -- 172.21.15.64:0/2189936160 shutdown_connections 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 --2- 172.21.15.64:0/2189936160 >> v2:172.21.15.100:3300/0 conn(0x7fd5500b2f20 0x7fd55006c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd5500cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 --2- 172.21.15.64:0/2189936160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55015e250 0x7fd5500ce4e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 -- 172.21.15.64:0/2189936160 >> 172.21.15.64:0/2189936160 conn(0x7fd5500d1c60 msgr2=0x7fd5500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 -- 172.21.15.64:0/2189936160 shutdown_connections 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.117+0000 7fd557c15640 1 -- 172.21.15.64:0/2189936160 wait complete. 2024-04-01T00:14:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.118+0000 7fd557c15640 1 Processor -- start 2024-04-01T00:14:17.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.118+0000 7fd557c15640 1 -- start start 2024-04-01T00:14:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd550160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd55015e250 0x7fd5500c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd55002ce60 con 0x7fd5500b2f20 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd55002cce0 con 0x7fd55015e250 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd557c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd55015e630 con 0x7fd5500d2780 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd5559b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd5559b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33246/0 (socket says 172.21.15.64:33246) 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd5559b1640 1 -- 172.21.15.64:0/3165641259 learned_addr learned my addr 172.21.15.64:0/3165641259 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd5551b0640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd550160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.119+0000 7fd5561b2640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd55015e250 0x7fd5500c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 -- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 msgr2=0x7fd550160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd550160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 -- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd55015e250 msgr2=0x7fd5500c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd55015e250 0x7fd5500c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 -- 172.21.15.64:0/3165641259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55015f5c0 con 0x7fd5500b2f20 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd5559b1640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548032670 con 0x7fd5500b2f20 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd548032810 con 0x7fd5500b2f20 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548032c00 con 0x7fd5500b2f20 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55015f8a0 con 0x7fd5500b2f20 2024-04-01T00:14:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.120+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55015fe60 con 0x7fd5500b2f20 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.121+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5500681a0 con 0x7fd5500b2f20 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.121+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd54804c060 con 0x7fd5500b2f20 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.122+0000 7fd54e7fc640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 0x7fd53c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.122+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5480ac020 con 0x7fd5500b2f20 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.122+0000 7fd5551b0640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 0x7fd53c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.122+0000 7fd5551b0640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 0x7fd53c06de50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5500c6e10 tx=0x7fd544009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.126+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd548040070 con 0x7fd5500b2f20 2024-04-01T00:14:17.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.305+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd550067910 con 0x7fd5500b2f20 2024-04-01T00:14:17.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.306+0000 7fd54e7fc640 1 -- 172.21.15.64:0/3165641259 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd54807a6a0 con 0x7fd5500b2f20 2024-04-01T00:14:17.307 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 msgr2=0x7fd53c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 0x7fd53c06de50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5500c6e10 tx=0x7fd544009190 comp rx=0 tx=0).stop 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 msgr2=0x7fd550156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd5480046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 shutdown_connections 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd53c06b990 0x7fd53c06de50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd55015e250 0x7fd5500c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5500d2780 0x7fd550160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 --2- 172.21.15.64:0/3165641259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5500b2f20 0x7fd550156ad0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 >> 172.21.15.64:0/3165641259 conn(0x7fd5500d1c60 msgr2=0x7fd5500c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:17.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 shutdown_connections 2024-04-01T00:14:17.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.308+0000 7fd557c15640 1 -- 172.21.15.64:0/3165641259 wait complete. 2024-04-01T00:14:17.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-01T00:14:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 Processor -- start 2024-04-01T00:14:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 -- start start 2024-04-01T00:14:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05801619e0 0x7f05800d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05800b3820 0x7f05800cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05800b3cb0 con 0x7f05800cf910 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f05800d2ba0 con 0x7f05801619e0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f0586672640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f05800d2e80 con 0x7f05800b3820 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f057ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05801619e0 0x7f05800d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f057ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05801619e0 0x7f05800d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33254/0 (socket says 172.21.15.64:33254) 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.421+0000 7f057ffff640 1 -- 172.21.15.64:0/2623031309 learned_addr learned my addr 172.21.15.64:0/2623031309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057f7fe640 1 --2- 172.21.15.64:0/2623031309 >> v2:172.21.15.100:3300/0 conn(0x7f05800b3820 0x7f05800cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057ffff640 1 -- 172.21.15.64:0/2623031309 >> v2:172.21.15.100:3300/0 conn(0x7f05800b3820 msgr2=0x7f05800cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057ffff640 1 --2- 172.21.15.64:0/2623031309 >> v2:172.21.15.100:3300/0 conn(0x7f05800b3820 0x7f05800cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057ffff640 1 -- 172.21.15.64:0/2623031309 >> v1:172.21.15.100:6789/0 conn(0x7f05800cf910 legacy=0x7f05800d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057ffff640 1 -- 172.21.15.64:0/2623031309 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05800ceeb0 con 0x7f05801619e0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.422+0000 7f057ffff640 1 --2- 172.21.15.64:0/2623031309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f05800d4a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f058008c0a0 tx=0x7f057002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d90ed9a157e219 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f057effd640 1 -- 172.21.15.64:0/2623031309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570035020 con 0x7f05801619e0 2024-04-01T00:14:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f057effd640 1 -- 172.21.15.64:0/2623031309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570031870 con 0x7f05801619e0 2024-04-01T00:14:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f057effd640 1 -- 172.21.15.64:0/2623031309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570031c40 con 0x7f05801619e0 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 -- 172.21.15.64:0/2623031309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 msgr2=0x7f05800d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 --2- 172.21.15.64:0/2623031309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f05800d4a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f058008c0a0 tx=0x7f057002b910 comp rx=0 tx=0).stop 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 -- 172.21.15.64:0/2623031309 shutdown_connections 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 --2- 172.21.15.64:0/2623031309 >> v2:172.21.15.100:3300/0 conn(0x7f05800b3820 0x7f05800cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 --2- 172.21.15.64:0/2623031309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f05800d4a60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 -- 172.21.15.64:0/2623031309 >> 172.21.15.64:0/2623031309 conn(0x7f058009d290 msgr2=0x7f05800d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 -- 172.21.15.64:0/2623031309 shutdown_connections 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.423+0000 7f0586672640 1 -- 172.21.15.64:0/2623031309 wait complete. 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.424+0000 7f0586672640 1 Processor -- start 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 -- start start 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05800b3820 0x7f0580160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 0x7f05800c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05800b31b0 con 0x7f05801619e0 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05800d2890 con 0x7f05800b3820 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0586672640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f05800ceb10 con 0x7f05800cf910 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0584c0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0584c0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33262/0 (socket says 172.21.15.64:33262) 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f0584c0f640 1 -- 172.21.15.64:0/1688201859 learned_addr learned my addr 172.21.15.64:0/1688201859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f057f7fe640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 0x7f05800c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.425+0000 7f057ffff640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05800b3820 0x7f0580160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 -- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05800b3820 msgr2=0x7f0580160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05800b3820 0x7f0580160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 -- 172.21.15.64:0/1688201859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 msgr2=0x7f058006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 -- 172.21.15.64:0/1688201859 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05800ca000 con 0x7f05800cf910 2024-04-01T00:14:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f0584c0f640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057f7fe640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 0x7f05800c8060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f056800e850 tx=0x7f056800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568017070 con 0x7f05800cf910 2024-04-01T00:14:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056800c940 con 0x7f05800cf910 2024-04-01T00:14:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056800cd10 con 0x7f05800cf910 2024-04-01T00:14:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05800ca2e0 con 0x7f05800cf910 2024-04-01T00:14:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.426+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05800ca8a0 con 0x7f05800cf910 2024-04-01T00:14:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.427+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05800681a0 con 0x7f05800cf910 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.427+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f056801a070 con 0x7f05800cf910 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.428+0000 7f057cff9640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 0x7f056006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.428+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f056808ab20 con 0x7f05800cf910 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.428+0000 7f057ffff640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 0x7f056006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.428+0000 7f057ffff640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 0x7f056006de50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0570036000 tx=0x7f057003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.432+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0568059060 con 0x7f05800cf910 2024-04-01T00:14:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:17.681+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7f05800ce470 con 0x7f05800cf910 2024-04-01T00:14:18.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.095+0000 7f057cff9640 1 -- 172.21.15.64:0/1688201859 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v50) ==== 147+0+0 (secure 0 0 0) 0x7f056805d100 con 0x7f05800cf910 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 msgr2=0x7f056006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 0x7f056006de50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0570036000 tx=0x7f057003c040 comp rx=0 tx=0).stop 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 msgr2=0x7f05800c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 0x7f05800c8060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f056800e850 tx=0x7f056800ed10 comp rx=0 tx=0).stop 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 shutdown_connections 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f056006b990 0x7f056006de50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05801619e0 0x7f058006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05800cf910 0x7f05800c8060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 --2- 172.21.15.64:0/1688201859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05800b3820 0x7f0580160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 >> 172.21.15.64:0/1688201859 conn(0x7f058009d290 msgr2=0x7f058015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:18.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 shutdown_connections 2024-04-01T00:14:18.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.098+0000 7f0586672640 1 -- 172.21.15.64:0/1688201859 wait complete. 2024-04-01T00:14:18.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-01T00:14:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 Processor -- start 2024-04-01T00:14:18.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 -- start start 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f700815e230 0x7f7008161d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70080cc5a0 0x7f70081622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70080d01a0 con 0x7f70081627e0 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f70080d0cc0 con 0x7f700815e230 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.216+0000 7f700d66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f700815e610 con 0x7f70080cc5a0 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f700815e230 0x7f7008161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f700815e230 0x7f7008161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33276/0 (socket says 172.21.15.64:33276) 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 -- 172.21.15.64:0/3700160170 learned_addr learned my addr 172.21.15.64:0/3700160170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f6ffffff640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70080cc5a0 0x7f70081622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 -- 172.21.15.64:0/3700160170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70080cc5a0 msgr2=0x7f70081622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70080cc5a0 0x7f70081622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 -- 172.21.15.64:0/3700160170 >> v1:172.21.15.100:6789/0 conn(0x7f70081627e0 legacy=0x7f70080cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.217+0000 7f7006ffd640 1 -- 172.21.15.64:0/3700160170 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70080d0420 con 0x7f700815e230 2024-04-01T00:14:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f7006ffd640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700815e230 0x7f7008161d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567dd6c0e546361b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f70067fc640 1 -- 172.21.15.64:0/3700160170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8034070 con 0x7f700815e230 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f70067fc640 1 -- 172.21.15.64:0/3700160170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff8030840 con 0x7f700815e230 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f70067fc640 1 -- 172.21.15.64:0/3700160170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8030c10 con 0x7f700815e230 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 -- 172.21.15.64:0/3700160170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700815e230 msgr2=0x7f7008161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700815e230 0x7f7008161d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff802b980 comp rx=0 tx=0).stop 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 -- 172.21.15.64:0/3700160170 shutdown_connections 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70080cc5a0 0x7f70081622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 --2- 172.21.15.64:0/3700160170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700815e230 0x7f7008161d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.218+0000 7f700d66b640 1 -- 172.21.15.64:0/3700160170 >> 172.21.15.64:0/3700160170 conn(0x7f70080719a0 msgr2=0x7f70080b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.219+0000 7f700d66b640 1 -- 172.21.15.64:0/3700160170 shutdown_connections 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.219+0000 7f700d66b640 1 -- 172.21.15.64:0/3700160170 wait complete. 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 Processor -- start 2024-04-01T00:14:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 -- start start 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70080cc5a0 0x7f70081f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f700815e230 0x7f70080b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70080cbd00 con 0x7f70081627e0 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70080d09e0 con 0x7f70080cc5a0 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f700d66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f70080cbe80 con 0x7f700815e230 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.220+0000 7f70077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33292/0 (socket says 172.21.15.64:33292) 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 -- 172.21.15.64:0/293865240 learned_addr learned my addr 172.21.15.64:0/293865240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f7006ffd640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70080cc5a0 0x7f70081f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f6ffffff640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f700815e230 0x7f70080b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 -- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f700815e230 msgr2=0x7f70080b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f700815e230 0x7f70080b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 -- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70080cc5a0 msgr2=0x7f70081f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70080cc5a0 0x7f70081f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 -- 172.21.15.64:0/293865240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700815f190 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f70077fe640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f700000c730 tx=0x7f700000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000003b70 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7000003d10 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700815f470 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700000aa90 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.221+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70080b4000 con 0x7f70081627e0 2024-04-01T00:14:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.222+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70080681a0 con 0x7f70081627e0 2024-04-01T00:14:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.223+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f700000ac30 con 0x7f70081627e0 2024-04-01T00:14:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.223+0000 7f6fff7fe640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 0x7f6fe806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.223+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f700008a610 con 0x7f70081627e0 2024-04-01T00:14:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.223+0000 7f7006ffd640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 0x7f6fe806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.224+0000 7f7006ffd640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 0x7f6fe806de50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8036000 tx=0x7f6ff8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.228+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f70080d01a0 con 0x7f70081627e0 2024-04-01T00:14:18.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.407+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f70080af6b0 con 0x7f70081627e0 2024-04-01T00:14:18.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.410+0000 7f6fff7fe640 1 -- 172.21.15.64:0/293865240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f7000058bd0 con 0x7f70081627e0 2024-04-01T00:14:18.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 msgr2=0x7f6fe806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 0x7f6fe806de50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8036000 tx=0x7f6ff8008030 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 msgr2=0x7f70080b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f700000c730 tx=0x7f700000cbf0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 shutdown_connections 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6fe806b990 0x7f6fe806de50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70081627e0 0x7f70080b0e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f700815e230 0x7f70080b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 --2- 172.21.15.64:0/293865240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70080cc5a0 0x7f70081f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.412+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 >> 172.21.15.64:0/293865240 conn(0x7f70080719a0 msgr2=0x7f7008161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.413+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 shutdown_connections 2024-04-01T00:14:18.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.413+0000 7f700d66b640 1 -- 172.21.15.64:0/293865240 wait complete. 2024-04-01T00:14:18.424 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-01T00:14:18.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-01T00:14:18.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-01T00:14:18.502 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-01T00:14:18.502 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:18.503 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:18.518 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:18.518 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:18.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:18.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.713+0000 7fd5dbd25640 1 Processor -- start 2024-04-01T00:14:18.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.713+0000 7fd5dbd25640 1 -- start start 2024-04-01T00:14:18.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5dbd25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5d40d4a60 0x7fd5d40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5dbd25640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5d40cfae0 0x7fd5d41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5dbd25640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d40d2ba0 con 0x7fd5d41619e0 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5dbd25640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d40d2e80 con 0x7fd5d40cfae0 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5dbd25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d40ceeb0 con 0x7fd5d40d4a60 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5da2c2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5d40cfae0 0x7fd5d41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5da2c2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5d40cfae0 0x7fd5d41620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33294/0 (socket says 172.21.15.64:33294) 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5da2c2640 1 -- 172.21.15.64:0/3227626299 learned_addr learned my addr 172.21.15.64:0/3227626299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5d9ac1640 1 --1- 172.21.15.64:0/3227626299 >> v1:172.21.15.100:6789/0 conn(0x7fd5d41619e0 0x7fd5d40cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:43364/0 (socket says 172.21.15.64:43364) 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5d92c0640 1 --2- 172.21.15.64:0/3227626299 >> v2:172.21.15.100:3300/0 conn(0x7fd5d40d4a60 0x7fd5d40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5da2c2640 1 -- 172.21.15.64:0/3227626299 >> v2:172.21.15.100:3300/0 conn(0x7fd5d40d4a60 msgr2=0x7fd5d40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.714+0000 7fd5da2c2640 1 --2- 172.21.15.64:0/3227626299 >> v2:172.21.15.100:3300/0 conn(0x7fd5d40d4a60 0x7fd5d40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5da2c2640 1 -- 172.21.15.64:0/3227626299 >> v1:172.21.15.100:6789/0 conn(0x7fd5d41619e0 legacy=0x7fd5d40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5da2c2640 1 -- 172.21.15.64:0/3227626299 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d4161dc0 con 0x7fd5d40cfae0 2024-04-01T00:14:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5d8abf640 1 -- 172.21.15.64:0/3227626299 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268344623 0 0) 0x7fd5d40d2ba0 con 0x7fd5d41619e0 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5da2c2640 1 --2- 172.21.15.64:0/3227626299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d40cfae0 0x7fd5d41620a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5d408c0a0 tx=0x7fd5c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ef7a10f0b04ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5d8abf640 1 -- 172.21.15.64:0/3227626299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c402ca80 con 0x7fd5d40cfae0 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5d8abf640 1 -- 172.21.15.64:0/3227626299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402cc20 con 0x7fd5d40cfae0 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.715+0000 7fd5d8abf640 1 -- 172.21.15.64:0/3227626299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40343f0 con 0x7fd5d40cfae0 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 -- 172.21.15.64:0/3227626299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d40cfae0 msgr2=0x7fd5d41620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/3227626299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d40cfae0 0x7fd5d41620a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5d408c0a0 tx=0x7fd5c402bee0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 -- 172.21.15.64:0/3227626299 shutdown_connections 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/3227626299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d40cfae0 0x7fd5d41620a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/3227626299 >> v2:172.21.15.100:3300/0 conn(0x7fd5d40d4a60 0x7fd5d40b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 -- 172.21.15.64:0/3227626299 >> 172.21.15.64:0/3227626299 conn(0x7fd5d409d290 msgr2=0x7fd5d40d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 -- 172.21.15.64:0/3227626299 shutdown_connections 2024-04-01T00:14:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.716+0000 7fd5dbd25640 1 -- 172.21.15.64:0/3227626299 wait complete. 2024-04-01T00:14:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.717+0000 7fd5dbd25640 1 Processor -- start 2024-04-01T00:14:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.717+0000 7fd5dbd25640 1 -- start start 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cfae0 0x7fd5d40b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d40d4a60 0x7fd5d41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d40ceb10 con 0x7fd5d41619e0 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d40b31b0 con 0x7fd5d40cfae0 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5dbd25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d40cec90 con 0x7fd5d40d4a60 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33308/0 (socket says 172.21.15.64:33308) 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 -- 172.21.15.64:0/626679535 learned_addr learned my addr 172.21.15.64:0/626679535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5d9ac1640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cfae0 0x7fd5d40b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5d92c0640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d40d4a60 0x7fd5d41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 -- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d40d4a60 msgr2=0x7fd5d41f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d40d4a60 0x7fd5d41f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 -- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cfae0 msgr2=0x7fd5d40b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cfae0 0x7fd5d40b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 -- 172.21.15.64:0/626679535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d40b6bd0 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.718+0000 7fd5da2c2640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f40 tx=0x7fd5c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.719+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4043020 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.719+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c403e450 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.719+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4009510 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.719+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41f4580 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.719+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41f4b40 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.720+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d40681a0 con 0x7fd5d41619e0 2024-04-01T00:14:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.720+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd5c4046020 con 0x7fd5d41619e0 2024-04-01T00:14:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.720+0000 7fd5d27fc640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 0x7fd5c006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.721+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5c40ace50 con 0x7fd5d41619e0 2024-04-01T00:14:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.721+0000 7fd5d9ac1640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 0x7fd5c006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.721+0000 7fd5d9ac1640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 0x7fd5c006de50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc007fd0 tx=0x7fd5cc007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.725+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd5c404d030 con 0x7fd5d41619e0 2024-04-01T00:14:18.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.906+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5d40cfff0 con 0x7fd5d41619e0 2024-04-01T00:14:18.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.907+0000 7fd5d27fc640 1 -- 172.21.15.64:0/626679535 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fd5c407b410 con 0x7fd5d41619e0 2024-04-01T00:14:18.908 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:18.908 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:18.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 msgr2=0x7fd5c006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 0x7fd5c006de50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc007fd0 tx=0x7fd5cc007de0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 msgr2=0x7fd5d40c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:18.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f40 tx=0x7fd5c4004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 shutdown_connections 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd5c006b990 0x7fd5c006de50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5d41619e0 0x7fd5d40c7ac0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d40d4a60 0x7fd5d41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 --2- 172.21.15.64:0/626679535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cfae0 0x7fd5d40b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 >> 172.21.15.64:0/626679535 conn(0x7fd5d409d290 msgr2=0x7fd5d40d3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 shutdown_connections 2024-04-01T00:14:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:18.909+0000 7fd5dbd25640 1 -- 172.21.15.64:0/626679535 wait complete. 2024-04-01T00:14:18.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:18.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:19.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.025+0000 7f124626f640 1 Processor -- start 2024-04-01T00:14:19.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.025+0000 7f124626f640 1 -- start start 2024-04-01T00:14:19.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124626f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 0x7f124015be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124626f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f124006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124626f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1240071360 con 0x7f124006eb30 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124626f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f124006ef80 con 0x7f1240158a70 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124626f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1240071640 con 0x7f1240070bf0 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124480c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f124006eb30 0x7f124015e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49784/0 (socket says 172.21.15.64:49784) 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f124480c640 1 -- 172.21.15.64:0/2583417910 learned_addr learned my addr 172.21.15.64:0/2583417910 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123ffff640 1 --2- 172.21.15.64:0/2583417910 >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 0x7f124015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123f7fe640 1 --2- 172.21.15.64:0/2583417910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f124006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123effd640 1 -- 172.21.15.64:0/2583417910 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260259484 0 0) 0x7f1240071360 con 0x7f124006eb30 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123effd640 1 -- 172.21.15.64:0/2583417910 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1228003520 con 0x7f124006eb30 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123f7fe640 1 -- 172.21.15.64:0/2583417910 >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 msgr2=0x7f124015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123f7fe640 1 --2- 172.21.15.64:0/2583417910 >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 0x7f124015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123f7fe640 1 -- 172.21.15.64:0/2583417910 >> v1:172.21.15.64:6789/0 conn(0x7f124006eb30 legacy=0x7f124015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123f7fe640 1 -- 172.21.15.64:0/2583417910 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1240071920 con 0x7f1240070bf0 2024-04-01T00:14:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.026+0000 7f123ffff640 1 --2- 172.21.15.64:0/2583417910 >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 0x7f124015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.027+0000 7f123f7fe640 1 --2- 172.21.15.64:0/2583417910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f124006e5f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12340097a0 tx=0x7f123402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c34190f2fe718e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f123effd640 1 -- 172.21.15.64:0/2583417910 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234035020 con 0x7f1240070bf0 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f123effd640 1 -- 172.21.15.64:0/2583417910 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1234031810 con 0x7f1240070bf0 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f123effd640 1 -- 172.21.15.64:0/2583417910 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234031be0 con 0x7f1240070bf0 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 -- 172.21.15.64:0/2583417910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 msgr2=0x7f124006e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 --2- 172.21.15.64:0/2583417910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f124006e5f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12340097a0 tx=0x7f123402b8b0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 -- 172.21.15.64:0/2583417910 shutdown_connections 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 --2- 172.21.15.64:0/2583417910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f124006e5f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 --2- 172.21.15.64:0/2583417910 >> v2:172.21.15.100:3300/0 conn(0x7f1240158a70 0x7f124015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 -- 172.21.15.64:0/2583417910 >> 172.21.15.64:0/2583417910 conn(0x7f12400b1730 msgr2=0x7f12400b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 -- 172.21.15.64:0/2583417910 shutdown_connections 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.028+0000 7f124626f640 1 -- 172.21.15.64:0/2583417910 wait complete. 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.029+0000 7f124626f640 1 Processor -- start 2024-04-01T00:14:19.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 -- start start 2024-04-01T00:14:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f12400b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1240158a70 0x7f12401677b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f12400b4170 con 0x7f124006eb30 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1240071070 con 0x7f1240158a70 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124626f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f12400b42f0 con 0x7f1240070bf0 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f123ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f123ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33322/0 (socket says 172.21.15.64:33322) 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f123ffff640 1 -- 172.21.15.64:0/1566980427 learned_addr learned my addr 172.21.15.64:0/1566980427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f123f7fe640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f12400b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.030+0000 7f124480c640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1240158a70 0x7f12401677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 -- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 msgr2=0x7f12400b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f12400b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 -- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1240158a70 msgr2=0x7f12401677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1240158a70 0x7f12401677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 -- 172.21.15.64:0/1566980427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12400ae200 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123ffff640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f123000e990 tx=0x7f123000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230017070 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123000c9e0 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123000cdb0 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12400ae4a0 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.031+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12400aea60 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.032+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12400681a0 con 0x7f124006eb30 2024-04-01T00:14:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.033+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f123001a070 con 0x7f124006eb30 2024-04-01T00:14:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.033+0000 7f123cff9640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 0x7f122c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.033+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f123008a9e0 con 0x7f124006eb30 2024-04-01T00:14:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.033+0000 7f123f7fe640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 0x7f122c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.034+0000 7f123f7fe640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 0x7f122c06de50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1234036000 tx=0x7f1234008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.038+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1230058fa0 con 0x7f124006eb30 2024-04-01T00:14:19.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.219+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f124015e1b0 con 0x7f124006eb30 2024-04-01T00:14:19.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.219+0000 7f123cff9640 1 -- 172.21.15.64:0/1566980427 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f123005d040 con 0x7f124006eb30 2024-04-01T00:14:19.220 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 msgr2=0x7f122c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 0x7f122c06de50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1234036000 tx=0x7f1234008030 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 msgr2=0x7f12400b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f123000e990 tx=0x7f123000ee50 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 shutdown_connections 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f122c06b990 0x7f122c06de50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1240158a70 0x7f12401677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1240070bf0 0x7f12400b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 --2- 172.21.15.64:0/1566980427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124006eb30 0x7f12400b3020 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 >> 172.21.15.64:0/1566980427 conn(0x7f12400b1730 msgr2=0x7f12400b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 shutdown_connections 2024-04-01T00:14:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.221+0000 7f124626f640 1 -- 172.21.15.64:0/1566980427 wait complete. 2024-04-01T00:14:19.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.337+0000 7f946fb3e640 1 Processor -- start 2024-04-01T00:14:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.337+0000 7f946fb3e640 1 -- start start 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946fb3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 0x7f946806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946fb3e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94680cb020 con 0x7f9468144720 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946fb3e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94680cb300 con 0x7f946806ed00 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946fb3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9468144b00 con 0x7f94680cab60 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946d8da640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9468144720 0x7f946806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49792/0 (socket says 172.21.15.64:49792) 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946d8da640 1 -- 172.21.15.64:0/2659246291 learned_addr learned my addr 172.21.15.64:0/2659246291 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946e0db640 1 --2- 172.21.15.64:0/2659246291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 0x7f946806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.338+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853433089 0 0) 0x7f94680cb020 con 0x7f9468144720 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9450003520 con 0x7f9468144720 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447142016 0 0) 0x7f94680cb300 con 0x7f946806ed00 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94680cb020 con 0x7f946806ed00 2024-04-01T00:14:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946e0db640 1 -- 172.21.15.64:0/2659246291 >> v1:172.21.15.100:6789/0 conn(0x7f946806ed00 legacy=0x7f9468070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946e0db640 1 -- 172.21.15.64:0/2659246291 >> v1:172.21.15.64:6789/0 conn(0x7f9468144720 legacy=0x7f946806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946e0db640 1 -- 172.21.15.64:0/2659246291 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94680cb5e0 con 0x7f94680cab60 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.339+0000 7f946e0db640 1 --2- 172.21.15.64:0/2659246291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 0x7f946806ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94600097d0 tx=0x7f946002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403563fae1995e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460035020 con 0x7f94680cab60 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9460031890 con 0x7f94680cab60 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946c8d8640 1 -- 172.21.15.64:0/2659246291 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460031c60 con 0x7f94680cab60 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 -- 172.21.15.64:0/2659246291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 msgr2=0x7f946806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2659246291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 0x7f946806ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94600097d0 tx=0x7f946002b930 comp rx=0 tx=0).stop 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 -- 172.21.15.64:0/2659246291 shutdown_connections 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2659246291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94680cab60 0x7f946806ce70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 -- 172.21.15.64:0/2659246291 >> 172.21.15.64:0/2659246291 conn(0x7f94680cfa90 msgr2=0x7f94680cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 -- 172.21.15.64:0/2659246291 shutdown_connections 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.340+0000 7f946fb3e640 1 -- 172.21.15.64:0/2659246291 wait complete. 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.341+0000 7f946fb3e640 1 Processor -- start 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.341+0000 7f946fb3e640 1 -- start start 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 0x7f94680c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94680cab60 0x7f946815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94680cd7f0 con 0x7f9468144720 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94680cbba0 con 0x7f94680cab60 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946fb3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f94680cd970 con 0x7f946806ed00 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33328/0 (socket says 172.21.15.64:33328) 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 -- 172.21.15.64:0/2138445843 learned_addr learned my addr 172.21.15.64:0/2138445843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946d8da640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 0x7f94680c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946d0d9640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94680cab60 0x7f946815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 -- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 msgr2=0x7f94680c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 0x7f94680c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 -- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94680cab60 msgr2=0x7f946815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.342+0000 7f946e0db640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94680cab60 0x7f946815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f946e0db640 1 -- 172.21.15.64:0/2138445843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94680c7e50 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f946d8da640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 0x7f94680c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f946e0db640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f94600097d0 tx=0x7f94600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460035040 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9460030ca0 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94680c7890 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460006e10 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.343+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94680b39c0 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.344+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94680681a0 con 0x7f9468144720 2024-04-01T00:14:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.344+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f9460038020 con 0x7f9468144720 2024-04-01T00:14:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.345+0000 7f94667fc640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 0x7f945406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.345+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94600accf0 con 0x7f9468144720 2024-04-01T00:14:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.345+0000 7f946d8da640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 0x7f945406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.345+0000 7f946d8da640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 0x7f945406de50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94580012e0 tx=0x7f9458001ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.349+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f94600b1070 con 0x7f9468144720 2024-04-01T00:14:19.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.529+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9468141ad0 con 0x7f9468144720 2024-04-01T00:14:19.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.529+0000 7f94667fc640 1 -- 172.21.15.64:0/2138445843 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f946002d040 con 0x7f9468144720 2024-04-01T00:14:19.531 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.531+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 msgr2=0x7f945406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.531+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 0x7f945406de50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94580012e0 tx=0x7f9458001ad0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.531+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 msgr2=0x7f9468142e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.531+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f94600097d0 tx=0x7f94600046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 shutdown_connections 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f945406b990 0x7f945406de50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9468144720 0x7f9468142e90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94680cab60 0x7f946815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 --2- 172.21.15.64:0/2138445843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f946806ed00 0x7f94680c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 >> 172.21.15.64:0/2138445843 conn(0x7f94680cfa90 msgr2=0x7f9468072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 shutdown_connections 2024-04-01T00:14:19.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.532+0000 7f946fb3e640 1 -- 172.21.15.64:0/2138445843 wait complete. 2024-04-01T00:14:19.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.647+0000 7f770fb93640 1 Processor -- start 2024-04-01T00:14:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.647+0000 7f770fb93640 1 -- start start 2024-04-01T00:14:19.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770fb93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770fb93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 0x7f77080ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770fb93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77080b3a10 con 0x7f77080b3500 2024-04-01T00:14:19.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770fb93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f770806cc50 con 0x7f7708160210 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770fb93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f77080b3cf0 con 0x7f77081599b0 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770e130640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 0x7f77080ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:57086/0 (socket says 172.21.15.64:57086) 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d12e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f77080b3500 0x7f770815fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:43382/0 (socket says 172.21.15.64:43382) 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 -- 172.21.15.64:0/1472735448 learned_addr learned my addr 172.21.15.64:0/1472735448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 -- 172.21.15.64:0/1472735448 >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 msgr2=0x7f77080ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:19.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 --2- 172.21.15.64:0/1472735448 >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 0x7f77080ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 -- 172.21.15.64:0/1472735448 >> v1:172.21.15.100:6789/0 conn(0x7f77080b3500 legacy=0x7f770815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770d92f640 1 -- 172.21.15.64:0/1472735448 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77080b3fd0 con 0x7f77081599b0 2024-04-01T00:14:19.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.648+0000 7f770e130640 1 --2- 172.21.15.64:0/1472735448 >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 0x7f77080ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.649+0000 7f770d92f640 1 --2- 172.21.15.64:0/1472735448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7700009870 tx=0x7f770002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345e6207ef732754 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.649+0000 7f770c92d640 1 -- 172.21.15.64:0/1472735448 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700034070 con 0x7f77081599b0 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.649+0000 7f770c92d640 1 -- 172.21.15.64:0/1472735448 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700030840 con 0x7f77081599b0 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.649+0000 7f770c92d640 1 -- 172.21.15.64:0/1472735448 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700030c10 con 0x7f77081599b0 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 -- 172.21.15.64:0/1472735448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 msgr2=0x7f77080b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 --2- 172.21.15.64:0/1472735448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7700009870 tx=0x7f770002b980 comp rx=0 tx=0).stop 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 -- 172.21.15.64:0/1472735448 shutdown_connections 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 --2- 172.21.15.64:0/1472735448 >> v2:172.21.15.100:3300/0 conn(0x7f7708160210 0x7f77080ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 --2- 172.21.15.64:0/1472735448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080b2fc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 -- 172.21.15.64:0/1472735448 >> 172.21.15.64:0/1472735448 conn(0x7f77080b1730 msgr2=0x7f77080b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 -- 172.21.15.64:0/1472735448 shutdown_connections 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.650+0000 7f770fb93640 1 -- 172.21.15.64:0/1472735448 wait complete. 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.651+0000 7f770fb93640 1 Processor -- start 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.651+0000 7f770fb93640 1 -- start start 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7708160210 0x7f770815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7708159d90 con 0x7f77080b3500 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77080ce280 con 0x7f7708160210 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770fb93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f77080aa740 con 0x7f77081599b0 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d12e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33342/0 (socket says 172.21.15.64:33342) 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 -- 172.21.15.64:0/4066558165 learned_addr learned my addr 172.21.15.64:0/4066558165 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770e130640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7708160210 0x7f770815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 -- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 msgr2=0x7f77080aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 -- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7708160210 msgr2=0x7f770815e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7708160210 0x7f770815e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 -- 172.21.15.64:0/4066558165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77080aaa20 con 0x7f77080b3500 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d12e640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.652+0000 7f770d92f640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f7700004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.653+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700048070 con 0x7f77080b3500 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.653+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700030640 con 0x7f77080b3500 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.653+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700034040 con 0x7f77080b3500 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.653+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77080aad00 con 0x7f77080b3500 2024-04-01T00:14:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.653+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77080ab2c0 con 0x7f77080b3500 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.654+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77080681a0 con 0x7f77080b3500 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.654+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f770004b070 con 0x7f77080b3500 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.654+0000 7f76fe7fc640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 0x7f76ec06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.654+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77000ac540 con 0x7f77080b3500 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.655+0000 7f770d12e640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 0x7f76ec06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.655+0000 7f770d12e640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 0x7f76ec06de50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f770815f430 tx=0x7f76f8007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.659+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f770007ab90 con 0x7f77080b3500 2024-04-01T00:14:19.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.839+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77080b4f30 con 0x7f77080b3500 2024-04-01T00:14:19.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.839+0000 7f76fe7fc640 1 -- 172.21.15.64:0/4066558165 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f770007ec30 con 0x7f77080b3500 2024-04-01T00:14:19.840 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 msgr2=0x7f76ec06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 0x7f76ec06de50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f770815f430 tx=0x7f76f8007e80 comp rx=0 tx=0).stop 2024-04-01T00:14:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 msgr2=0x7f77081434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f7700004730 comp rx=0 tx=0).stop 2024-04-01T00:14:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 shutdown_connections 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f76ec06b990 0x7f76ec06de50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7708160210 0x7f770815e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77081599b0 0x7f77080aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 --2- 172.21.15.64:0/4066558165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77080b3500 0x7f77081434f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 >> 172.21.15.64:0/4066558165 conn(0x7f77080b1730 msgr2=0x7f770815b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 shutdown_connections 2024-04-01T00:14:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.841+0000 7f770fb93640 1 -- 172.21.15.64:0/4066558165 wait complete. 2024-04-01T00:14:19.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:19.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.955+0000 7fc7d12d4640 1 Processor -- start 2024-04-01T00:14:19.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.955+0000 7fc7d12d4640 1 -- start start 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7d12d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc161a30 0x7fc7cc0d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7d12d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7d12d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7cc161e10 con 0x7fc7cc0b3740 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7d12d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fc7cc0b3ba0 con 0x7fc7cc0d0900 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7d12d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc7cc0cef20 con 0x7fc7cc161a30 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7cb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7ca7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc7cc0b3740 0x7fc7cc0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49796/0 (socket says 172.21.15.64:49796) 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7cb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33344/0 (socket says 172.21.15.64:33344) 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7ca7fc640 1 -- 172.21.15.64:0/1087373877 learned_addr learned my addr 172.21.15.64:0/1087373877 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.956+0000 7fc7caffd640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc161a30 0x7fc7cc0d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7cb7fe640 1 -- 172.21.15.64:0/1087373877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc161a30 msgr2=0x7fc7cc0d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7cb7fe640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc161a30 0x7fc7cc0d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7cb7fe640 1 -- 172.21.15.64:0/1087373877 >> v1:172.21.15.64:6789/0 conn(0x7fc7cc0b3740 legacy=0x7fc7cc0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7cb7fe640 1 -- 172.21.15.64:0/1087373877 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc0d2880 con 0x7fc7cc0d0900 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7cb7fe640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097a0 tx=0x7fc7c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354fbe66377c371d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7c9ffb640 1 -- 172.21.15.64:0/1087373877 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4035020 con 0x7fc7cc0d0900 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.957+0000 7fc7c9ffb640 1 -- 172.21.15.64:0/1087373877 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c4031830 con 0x7fc7cc0d0900 2024-04-01T00:14:19.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7c9ffb640 1 -- 172.21.15.64:0/1087373877 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4031c00 con 0x7fc7cc0d0900 2024-04-01T00:14:19.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 -- 172.21.15.64:0/1087373877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0d0900 msgr2=0x7fc7cc0d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097a0 tx=0x7fc7c402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 -- 172.21.15.64:0/1087373877 shutdown_connections 2024-04-01T00:14:19.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0d0900 0x7fc7cc0d0db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/1087373877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc161a30 0x7fc7cc0d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 -- 172.21.15.64:0/1087373877 >> 172.21.15.64:0/1087373877 conn(0x7fc7cc09d250 msgr2=0x7fc7cc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 -- 172.21.15.64:0/1087373877 shutdown_connections 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.958+0000 7fc7d12d4640 1 -- 172.21.15.64:0/1087373877 wait complete. 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.959+0000 7fc7d12d4640 1 Processor -- start 2024-04-01T00:14:19.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.959+0000 7fc7d12d4640 1 -- start start 2024-04-01T00:14:19.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc0d0900 0x7fc7cc0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7cc161a30 0x7fc7cc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc0ceb40 con 0x7fc7cc0b3740 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc0d2e00 con 0x7fc7cc161a30 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7d12d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc7cc0cf3b0 con 0x7fc7cc0d0900 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33348/0 (socket says 172.21.15.64:33348) 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 -- 172.21.15.64:0/2884000965 learned_addr learned my addr 172.21.15.64:0/2884000965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7ca7fc640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc0d0900 0x7fc7cc0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7cb7fe640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7cc161a30 0x7fc7cc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 -- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc0d0900 msgr2=0x7fc7cc0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc0d0900 0x7fc7cc0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 -- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7cc161a30 msgr2=0x7fc7cc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7cc161a30 0x7fc7cc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.960+0000 7fc7caffd640 1 -- 172.21.15.64:0/2884000965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc15f3a0 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7caffd640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009d10 tx=0x7fc7bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc018020 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7bc014480 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc15f680 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc014830 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.961+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc15fc40 con 0x7fc7cc0b3740 2024-04-01T00:14:19.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.962+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7cc0ca270 con 0x7fc7cc0b3740 2024-04-01T00:14:19.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.962+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fc7bc01b020 con 0x7fc7cc0b3740 2024-04-01T00:14:19.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.963+0000 7fc7c37fe640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 0x7fc7ac06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:19.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.963+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7bc08b180 con 0x7fc7cc0b3740 2024-04-01T00:14:19.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.963+0000 7fc7ca7fc640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 0x7fc7ac06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:19.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.963+0000 7fc7ca7fc640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 0x7fc7ac06de50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc1f7870 tx=0x7fc7b4007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:19.967+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fc7cc0ca270 con 0x7fc7cc0b3740 2024-04-01T00:14:20.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.164+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7cc0681a0 con 0x7fc7cc0b3740 2024-04-01T00:14:20.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.165+0000 7fc7c37fe640 1 -- 172.21.15.64:0/2884000965 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc7bc022030 con 0x7fc7cc0b3740 2024-04-01T00:14:20.166 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 msgr2=0x7fc7ac06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 0x7fc7ac06de50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc1f7870 tx=0x7fc7b4007e80 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 msgr2=0x7fc7cc160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009d10 tx=0x7fc7bc00c490 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 shutdown_connections 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fc7ac06b990 0x7fc7ac06de50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7cc161a30 0x7fc7cc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7cc0d0900 0x7fc7cc0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 --2- 172.21.15.64:0/2884000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7cc0b3740 0x7fc7cc160440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 >> 172.21.15.64:0/2884000965 conn(0x7fc7cc09d250 msgr2=0x7fc7cc0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 shutdown_connections 2024-04-01T00:14:20.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.167+0000 7fc7d12d4640 1 -- 172.21.15.64:0/2884000965 wait complete. 2024-04-01T00:14:20.178 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:20.178 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:20.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:20.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.336+0000 7f2d5d765640 1 Processor -- start 2024-04-01T00:14:20.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.336+0000 7f2d5d765640 1 -- start start 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d5d765640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d580d1920 0x7f2d580d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d5d765640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d5806c3e0 0x7f2d580d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d5d765640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d580d2780 con 0x7f2d5815e3f0 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d5d765640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f2d580cf3a0 con 0x7f2d580d1920 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d5d765640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2d580d2240 con 0x7f2d5806c3e0 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d580d1920 0x7f2d580d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d580d1920 0x7f2d580d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33350/0 (socket says 172.21.15.64:33350) 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 -- 172.21.15.64:0/3994031021 learned_addr learned my addr 172.21.15.64:0/3994031021 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d577fe640 1 --2- 172.21.15.64:0/3994031021 >> v2:172.21.15.100:3300/0 conn(0x7f2d5806c3e0 0x7f2d580d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 -- 172.21.15.64:0/3994031021 >> v2:172.21.15.100:3300/0 conn(0x7f2d5806c3e0 msgr2=0x7f2d580d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/3994031021 >> v2:172.21.15.100:3300/0 conn(0x7f2d5806c3e0 0x7f2d580d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 -- 172.21.15.64:0/3994031021 >> v1:172.21.15.100:6789/0 conn(0x7f2d5815e3f0 legacy=0x7f2d58161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.337+0000 7f2d56ffd640 1 -- 172.21.15.64:0/3994031021 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580ce790 con 0x7f2d580d1920 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.338+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/3994031021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580d1920 0x7f2d580d1d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806ee40 tx=0x7f2d4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c150e014440e5890 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.338+0000 7f2d567fc640 1 -- 172.21.15.64:0/3994031021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48034070 con 0x7f2d580d1920 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.338+0000 7f2d567fc640 1 -- 172.21.15.64:0/3994031021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48030860 con 0x7f2d580d1920 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.338+0000 7f2d567fc640 1 -- 172.21.15.64:0/3994031021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48030c30 con 0x7f2d580d1920 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 -- 172.21.15.64:0/3994031021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580d1920 msgr2=0x7f2d580d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 --2- 172.21.15.64:0/3994031021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580d1920 0x7f2d580d1d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806ee40 tx=0x7f2d4802b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 -- 172.21.15.64:0/3994031021 shutdown_connections 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 --2- 172.21.15.64:0/3994031021 >> v2:172.21.15.100:3300/0 conn(0x7f2d5806c3e0 0x7f2d580d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 --2- 172.21.15.64:0/3994031021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580d1920 0x7f2d580d1d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 -- 172.21.15.64:0/3994031021 >> 172.21.15.64:0/3994031021 conn(0x7f2d5809a980 msgr2=0x7f2d5809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 -- 172.21.15.64:0/3994031021 shutdown_connections 2024-04-01T00:14:20.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.339+0000 7f2d5d765640 1 -- 172.21.15.64:0/3994031021 wait complete. 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.340+0000 7f2d5d765640 1 Processor -- start 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.340+0000 7f2d5d765640 1 -- start start 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d580d1920 0x7f2d580c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580b7100 0x7f2d580c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580cf030 con 0x7f2d5806c3e0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580ced50 con 0x7f2d580b7100 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2d580b0cd0 con 0x7f2d580d1920 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33354/0 (socket says 172.21.15.64:33354) 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 -- 172.21.15.64:0/981918071 learned_addr learned my addr 172.21.15.64:0/981918071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d4ffff640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d580d1920 0x7f2d580c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d577fe640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580b7100 0x7f2d580c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 -- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d580d1920 msgr2=0x7f2d580c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d580d1920 0x7f2d580c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 -- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580b7100 msgr2=0x7f2d580c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580b7100 0x7f2d580c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 -- 172.21.15.64:0/981918071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580c8710 con 0x7f2d5806c3e0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802b9e0 tx=0x7f2d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48048070 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48030660 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d581f41f0 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.341+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48034040 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.342+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d581f47b0 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.343+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d580681a0 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.343+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2d4804b070 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.343+0000 7f2d4f7fe640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 0x7f2d3806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.343+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d480ac960 con 0x7f2d5806c3e0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.344+0000 7f2d4ffff640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 0x7f2d3806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.344+0000 7f2d4ffff640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 0x7f2d3806de50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d5815d0d0 tx=0x7f2d40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.348+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2d580c8710 con 0x7f2d5806c3e0 2024-04-01T00:14:20.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.528+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d580b0a10 con 0x7f2d5806c3e0 2024-04-01T00:14:20.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.530+0000 7f2d4f7fe640 1 -- 172.21.15.64:0/981918071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f2d4807b0d0 con 0x7f2d5806c3e0 2024-04-01T00:14:20.531 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:20.531 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 msgr2=0x7f2d3806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 0x7f2d3806de50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d5815d0d0 tx=0x7f2d40009040 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 msgr2=0x7f2d580c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802b9e0 tx=0x7f2d48004730 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 shutdown_connections 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2d3806b990 0x7f2d3806de50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580b7100 0x7f2d580c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d580d1920 0x7f2d580c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 --2- 172.21.15.64:0/981918071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5806c3e0 0x7f2d580c58a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.531+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 >> 172.21.15.64:0/981918071 conn(0x7f2d5809a980 msgr2=0x7f2d580b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.532+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 shutdown_connections 2024-04-01T00:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.532+0000 7f2d5d765640 1 -- 172.21.15.64:0/981918071 wait complete. 2024-04-01T00:14:20.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:20.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:20.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.649+0000 7fec0a3d0640 1 Processor -- start 2024-04-01T00:14:20.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.649+0000 7fec0a3d0640 1 -- start start 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0a3d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0a3d0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec040cf400 con 0x7fec040b2f00 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0a3d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec040d0b80 con 0x7fec0415e640 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0a3d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec040d11d0 con 0x7fec0415e260 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0896d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fec040b2f00 0x7fec040cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49808/0 (socket says 172.21.15.64:49808) 2024-04-01T00:14:20.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec0896d640 1 -- 172.21.15.64:0/1836157015 learned_addr learned my addr 172.21.15.64:0/1836157015 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:20.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec03fff640 1 --2- 172.21.15.64:0/1836157015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec02ffd640 1 -- 172.21.15.64:0/1836157015 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027682795 0 0) 0x7fec040cf400 con 0x7fec040b2f00 2024-04-01T00:14:20.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec03fff640 1 -- 172.21.15.64:0/1836157015 >> v1:172.21.15.100:6789/0 conn(0x7fec0415e640 legacy=0x7fec040ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec03fff640 1 -- 172.21.15.64:0/1836157015 >> v1:172.21.15.64:6789/0 conn(0x7fec040b2f00 legacy=0x7fec040cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.650+0000 7fec03fff640 1 -- 172.21.15.64:0/1836157015 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec040d0e60 con 0x7fec0415e260 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.651+0000 7fec03fff640 1 --2- 172.21.15.64:0/1836157015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040d2890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febf4009750 tx=0x7febf402b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac311c4ae7d6ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.651+0000 7fec02ffd640 1 -- 172.21.15.64:0/1836157015 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4038070 con 0x7fec0415e260 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.651+0000 7fec02ffd640 1 -- 172.21.15.64:0/1836157015 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4031710 con 0x7fec0415e260 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.651+0000 7fec02ffd640 1 -- 172.21.15.64:0/1836157015 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4031ae0 con 0x7fec0415e260 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 -- 172.21.15.64:0/1836157015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 msgr2=0x7fec040d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/1836157015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040d2890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febf4009750 tx=0x7febf402b7b0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 -- 172.21.15.64:0/1836157015 shutdown_connections 2024-04-01T00:14:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/1836157015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040d2890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 -- 172.21.15.64:0/1836157015 >> 172.21.15.64:0/1836157015 conn(0x7fec040d1d70 msgr2=0x7fec040d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 -- 172.21.15.64:0/1836157015 shutdown_connections 2024-04-01T00:14:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.652+0000 7fec0a3d0640 1 -- 172.21.15.64:0/1836157015 wait complete. 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.653+0000 7fec0a3d0640 1 Processor -- start 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.653+0000 7fec0a3d0640 1 -- start start 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 0x7fec040b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cf110 con 0x7fec0415e640 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cef90 con 0x7fec040b2f00 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0a3d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec040ccc80 con 0x7fec0415e260 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 0x7fec040b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33360/0 (socket says 172.21.15.64:33360) 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 -- 172.21.15.64:0/2447962873 learned_addr learned my addr 172.21.15.64:0/2447962873 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec037fe640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 -- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 msgr2=0x7fec040b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 -- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 msgr2=0x7fec040b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 0x7fec040b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 -- 172.21.15.64:0/2447962873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec040b6280 con 0x7fec0415e640 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec03fff640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 0x7fec040b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec0896d640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c7d0 tx=0x7febfc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.654+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc018070 con 0x7fec0415e640 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.655+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc00a640 con 0x7fec0415e640 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.655+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec040b6560 con 0x7fec0415e640 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.655+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc00aa70 con 0x7fec0415e640 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.655+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec040b6e00 con 0x7fec0415e640 2024-04-01T00:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.656+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec040681a0 con 0x7fec0415e640 2024-04-01T00:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.656+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7febfc01b070 con 0x7fec0415e640 2024-04-01T00:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.656+0000 7fec00ff9640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 0x7febf006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.656+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febfc089e60 con 0x7fec0415e640 2024-04-01T00:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.657+0000 7fec03fff640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 0x7febf006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.657+0000 7fec03fff640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 0x7febf006de50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7febf4035000 tx=0x7febf4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.661+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7febfc058380 con 0x7fec0415e640 2024-04-01T00:14:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.842+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec040c9d90 con 0x7fec0415e640 2024-04-01T00:14:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.842+0000 7fec00ff9640 1 -- 172.21.15.64:0/2447962873 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7febfc05c420 con 0x7fec0415e640 2024-04-01T00:14:20.843 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 msgr2=0x7febf006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 0x7febf006de50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7febf4035000 tx=0x7febf4008030 comp rx=0 tx=0).stop 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 msgr2=0x7fec040c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c7d0 tx=0x7febfc00cc90 comp rx=0 tx=0).stop 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 shutdown_connections 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7febf006b990 0x7febf006de50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0415e640 0x7fec040c9850 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec0415e260 0x7fec040b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 --2- 172.21.15.64:0/2447962873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec040b2f00 0x7fec040b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 >> 172.21.15.64:0/2447962873 conn(0x7fec040d1d70 msgr2=0x7fec040d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 shutdown_connections 2024-04-01T00:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.844+0000 7fec0a3d0640 1 -- 172.21.15.64:0/2447962873 wait complete. 2024-04-01T00:14:20.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.959+0000 7f800fa0e640 1 Processor -- start 2024-04-01T00:14:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.960+0000 7f800fa0e640 1 -- start start 2024-04-01T00:14:20.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.960+0000 7f800fa0e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80080cae40 0x7f80080cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.960+0000 7f800fa0e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8008072dd0 con 0x7f80080ca350 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.960+0000 7f800fa0e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80080cb300 con 0x7f8008145290 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.960+0000 7f800fa0e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f80080cb5e0 con 0x7f80080cae40 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800d7aa640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f80080ca350 0x7f80080ca730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49822/0 (socket says 172.21.15.64:49822) 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800d7aa640 1 -- 172.21.15.64:0/294245497 learned_addr learned my addr 172.21.15.64:0/294245497 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800dfab640 1 --2- 172.21.15.64:0/294245497 >> v2:172.21.15.100:3300/0 conn(0x7f80080cae40 0x7f80080cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f8007fff640 1 -- 172.21.15.64:0/294245497 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883246247 0 0) 0x7f8008072dd0 con 0x7f80080ca350 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f8007fff640 1 -- 172.21.15.64:0/294245497 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff0003520 con 0x7f80080ca350 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800dfab640 1 -- 172.21.15.64:0/294245497 >> v1:172.21.15.100:6789/0 conn(0x7f8008145290 legacy=0x7f80081639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800dfab640 1 -- 172.21.15.64:0/294245497 >> v1:172.21.15.64:6789/0 conn(0x7f80080ca350 legacy=0x7f80080ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.961+0000 7f800dfab640 1 -- 172.21.15.64:0/294245497 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8008072910 con 0x7f80080cae40 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800dfab640 1 --2- 172.21.15.64:0/294245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080cbec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009d90 tx=0x7f7ff802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78009ff50240fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f8007fff640 1 -- 172.21.15.64:0/294245497 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8039070 con 0x7f80080cae40 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f8007fff640 1 -- 172.21.15.64:0/294245497 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff802cd00 con 0x7f80080cae40 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f8007fff640 1 -- 172.21.15.64:0/294245497 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8034660 con 0x7f80080cae40 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800fa0e640 1 -- 172.21.15.64:0/294245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 msgr2=0x7f80080cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800fa0e640 1 --2- 172.21.15.64:0/294245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080cbec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009d90 tx=0x7f7ff802bf50 comp rx=0 tx=0).stop 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800fa0e640 1 -- 172.21.15.64:0/294245497 shutdown_connections 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800fa0e640 1 --2- 172.21.15.64:0/294245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080cbec0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.962+0000 7f800fa0e640 1 -- 172.21.15.64:0/294245497 >> 172.21.15.64:0/294245497 conn(0x7f80080cfa90 msgr2=0x7f80080cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.963+0000 7f800fa0e640 1 -- 172.21.15.64:0/294245497 shutdown_connections 2024-04-01T00:14:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.963+0000 7f800fa0e640 1 -- 172.21.15.64:0/294245497 wait complete. 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 Processor -- start 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 -- start start 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8008145290 0x7f800815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8008167100 con 0x7f80080ca350 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8008071450 con 0x7f80080cae40 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.964+0000 7f800fa0e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8008167280 con 0x7f8008145290 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33362/0 (socket says 172.21.15.64:33362) 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 -- 172.21.15.64:0/4266294469 learned_addr learned my addr 172.21.15.64:0/4266294469 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800cfa9640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800dfab640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8008145290 0x7f800815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 -- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8008145290 msgr2=0x7f800815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8008145290 0x7f800815ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 -- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 msgr2=0x7f80080b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 -- 172.21.15.64:0/4266294469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080b4d30 con 0x7f80080ca350 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800d7aa640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f800000c440 tx=0x7f800000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000017070 con 0x7f80080ca350 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000005ce0 con 0x7f80080ca350 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80080b5010 con 0x7f80080ca350 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000072b0 con 0x7f80080ca350 2024-04-01T00:14:20.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.965+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80080b55d0 con 0x7f80080ca350 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.967+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80080b5810 con 0x7f80080ca350 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.967+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f800001a070 con 0x7f80080ca350 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.967+0000 7f8005ffb640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 0x7f7ff406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.967+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8000089a10 con 0x7f80080ca350 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.967+0000 7f800cfa9640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 0x7f7ff406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:20.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.968+0000 7f800cfa9640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 0x7f7ff406de80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80081656d0 tx=0x7f7ffc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:20.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:20.972+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8000058120 con 0x7f80080ca350 2024-04-01T00:14:21.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.152+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80080681a0 con 0x7f80080ca350 2024-04-01T00:14:21.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.153+0000 7f8005ffb640 1 -- 172.21.15.64:0/4266294469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f800005c1c0 con 0x7f80080ca350 2024-04-01T00:14:21.153 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:21.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 msgr2=0x7f7ff406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 0x7f7ff406de80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80081656d0 tx=0x7f7ffc009040 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 msgr2=0x7f80081660f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f800000c440 tx=0x7f800000c900 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 shutdown_connections 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ff406b9c0 0x7f7ff406de80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.154+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8008145290 0x7f800815ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.155+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80080cae40 0x7f80080b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.155+0000 7f800fa0e640 1 --2- 172.21.15.64:0/4266294469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80080ca350 0x7f80081660f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.155+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 >> 172.21.15.64:0/4266294469 conn(0x7f80080cfa90 msgr2=0x7f8008139b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.155+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 shutdown_connections 2024-04-01T00:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.155+0000 7f800fa0e640 1 -- 172.21.15.64:0/4266294469 wait complete. 2024-04-01T00:14:21.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:21.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.270+0000 7f79225a1640 1 Processor -- start 2024-04-01T00:14:21.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.270+0000 7f79225a1640 1 -- start start 2024-04-01T00:14:21.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f79225a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f79225a1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f791c15c3e0 0x7f791c0d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f79225a1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791c0b0360 con 0x7f791c0ad0f0 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f79225a1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f791c0d12e0 con 0x7f791c15c3e0 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f79225a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b0640 con 0x7f791c15c000 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f791bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f791b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f791c15c3e0 0x7f791c0d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f791bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:57162/0 (socket says 172.21.15.64:57162) 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f791b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f791c15c3e0 0x7f791c0d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:47320/0 (socket says 172.21.15.64:47320) 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f791bfff640 1 -- 172.21.15.64:0/119849285 learned_addr learned my addr 172.21.15.64:0/119849285 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.271+0000 7f7920b3e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f791c0ad0f0 0x7f791c0b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:43412/0 (socket says 172.21.15.64:43412) 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791b7fe640 1 -- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 msgr2=0x7f791c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791b7fe640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791b7fe640 1 -- 172.21.15.64:0/119849285 >> v1:172.21.15.100:6789/0 conn(0x7f791c0ad0f0 legacy=0x7f791c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791b7fe640 1 -- 172.21.15.64:0/119849285 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0b0920 con 0x7f791c15c3e0 2024-04-01T00:14:21.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791affd640 1 -- 172.21.15.64:0/119849285 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923622226 0 0) 0x7f791c0b0360 con 0x7f791c0ad0f0 2024-04-01T00:14:21.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791bfff640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791b7fe640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c15c3e0 0x7f791c0d0da0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f790c0097a0 tx=0x7f790c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96d9094aaabdc0f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791affd640 1 -- 172.21.15.64:0/119849285 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c039070 con 0x7f791c15c3e0 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791affd640 1 -- 172.21.15.64:0/119849285 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c0316f0 con 0x7f791c15c3e0 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.272+0000 7f791affd640 1 -- 172.21.15.64:0/119849285 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c031ac0 con 0x7f791c15c3e0 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 -- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c15c3e0 msgr2=0x7f791c0d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c15c3e0 0x7f791c0d0da0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f790c0097a0 tx=0x7f790c02b790 comp rx=0 tx=0).stop 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 -- 172.21.15.64:0/119849285 shutdown_connections 2024-04-01T00:14:21.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c15c3e0 0x7f791c0d0da0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 --2- 172.21.15.64:0/119849285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 -- 172.21.15.64:0/119849285 >> 172.21.15.64:0/119849285 conn(0x7f791c072810 msgr2=0x7f791c072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 -- 172.21.15.64:0/119849285 shutdown_connections 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.273+0000 7f79225a1640 1 -- 172.21.15.64:0/119849285 wait complete. 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.274+0000 7f79225a1640 1 Processor -- start 2024-04-01T00:14:21.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.274+0000 7f79225a1640 1 -- start start 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 0x7f791c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b1b10 con 0x7f791c15c3e0 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b0ee0 con 0x7f791c0ad0f0 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f79225a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b1c90 con 0x7f791c15c000 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f7920b3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f791bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 0x7f791c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f7920b3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33364/0 (socket says 172.21.15.64:33364) 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f791b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.275+0000 7f7920b3e640 1 -- 172.21.15.64:0/634819401 learned_addr learned my addr 172.21.15.64:0/634819401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:21.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 -- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 msgr2=0x7f791c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 -- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 msgr2=0x7f791c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 0x7f791c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 -- 172.21.15.64:0/634819401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c15e9e0 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f791b7fe640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f791bfff640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 0x7f791c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7920b3e640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f791400c730 tx=0x7f791400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914018070 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791400ce40 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791400a800 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c0b3b20 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.276+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c0b40e0 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.277+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c0681a0 con 0x7f791c15c3e0 2024-04-01T00:14:21.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.277+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f791401b070 con 0x7f791c15c3e0 2024-04-01T00:14:21.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.278+0000 7f7918ff9640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 0x7f790806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.278+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7914089c20 con 0x7f791c15c3e0 2024-04-01T00:14:21.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.278+0000 7f791bfff640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 0x7f790806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.279+0000 7f791bfff640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 0x7f790806de50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7910009630 tx=0x7f7910009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.282+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f791c15e9e0 con 0x7f791c15c3e0 2024-04-01T00:14:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.462+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f791c15ec10 con 0x7f791c15c3e0 2024-04-01T00:14:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.463+0000 7f7918ff9640 1 -- 172.21.15.64:0/634819401 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7914058270 con 0x7f791c15c3e0 2024-04-01T00:14:21.464 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.464+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 msgr2=0x7f790806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.464+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 0x7f790806de50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7910009630 tx=0x7f7910009240 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.464+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 msgr2=0x7f791c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.464+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f791400c730 tx=0x7f791400cbf0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 shutdown_connections 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f790806b990 0x7f790806de50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c15c3e0 0x7f791c0c94c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791c15c000 0x7f791c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 --2- 172.21.15.64:0/634819401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f791c0ad0f0 0x7f791c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 >> 172.21.15.64:0/634819401 conn(0x7f791c072810 msgr2=0x7f791c15b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 shutdown_connections 2024-04-01T00:14:21.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.465+0000 7f79225a1640 1 -- 172.21.15.64:0/634819401 wait complete. 2024-04-01T00:14:21.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:21.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.579+0000 7fa9667e5640 1 Processor -- start 2024-04-01T00:14:21.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.579+0000 7fa9667e5640 1 -- start start 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa9667e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9600ca350 0x7fa9600cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa9667e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa9667e5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa960072dd0 con 0x7fa960145290 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa9667e5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa9600cb300 con 0x7fa9600ca350 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa9667e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600cb5e0 con 0x7fa9600cae40 2024-04-01T00:14:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa960145290 0x7fa9600ca730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49828/0 (socket says 172.21.15.64:49828) 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95f7fe640 1 -- 172.21.15.64:0/2300880563 learned_addr learned my addr 172.21.15.64:0/2300880563 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95ffff640 1 --2- 172.21.15.64:0/2300880563 >> v2:172.21.15.100:3300/0 conn(0x7fa9600ca350 0x7fa9600cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa964d82640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95effd640 1 -- 172.21.15.64:0/2300880563 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685760214 0 0) 0x7fa960072dd0 con 0x7fa960145290 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95ffff640 1 -- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 msgr2=0x7fa9601659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95ffff640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95ffff640 1 -- 172.21.15.64:0/2300880563 >> v1:172.21.15.64:6789/0 conn(0x7fa960145290 legacy=0x7fa9600ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.580+0000 7fa95ffff640 1 -- 172.21.15.64:0/2300880563 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa960072910 con 0x7fa9600ca350 2024-04-01T00:14:21.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.581+0000 7fa95ffff640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600ca350 0x7fa9600cbec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097a0 tx=0x7fa94802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8fed1345eef7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.581+0000 7fa95effd640 1 -- 172.21.15.64:0/2300880563 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948035020 con 0x7fa9600ca350 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.581+0000 7fa95effd640 1 -- 172.21.15.64:0/2300880563 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa948031830 con 0x7fa9600ca350 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.581+0000 7fa95effd640 1 -- 172.21.15.64:0/2300880563 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948031c00 con 0x7fa9600ca350 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 -- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600ca350 msgr2=0x7fa9600cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600ca350 0x7fa9600cbec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097a0 tx=0x7fa94802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 -- 172.21.15.64:0/2300880563 shutdown_connections 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 --2- 172.21.15.64:0/2300880563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600ca350 0x7fa9600cbec0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 -- 172.21.15.64:0/2300880563 >> 172.21.15.64:0/2300880563 conn(0x7fa9600cfa90 msgr2=0x7fa9600cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 -- 172.21.15.64:0/2300880563 shutdown_connections 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.582+0000 7fa9667e5640 1 -- 172.21.15.64:0/2300880563 wait complete. 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.583+0000 7fa9667e5640 1 Processor -- start 2024-04-01T00:14:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.583+0000 7fa9667e5640 1 -- start start 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa960145290 0x7fa9600cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa960165f20 con 0x7fa9600ca350 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600713d0 con 0x7fa960145290 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa9667e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa9601660a0 con 0x7fa9600cae40 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33380/0 (socket says 172.21.15.64:33380) 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 -- 172.21.15.64:0/4018131927 learned_addr learned my addr 172.21.15.64:0/4018131927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:21.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95f7fe640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa964d82640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa960145290 0x7fa9600cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 -- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 msgr2=0x7fa9601557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 -- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa960145290 msgr2=0x7fa9600cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa960145290 0x7fa9600cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 -- 172.21.15.64:0/4018131927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601656e0 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95ffff640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098c0 tx=0x7fa9480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948040070 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.584+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94803e070 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.585+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa960155ce0 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.585+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948035040 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.585+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601f27a0 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.585+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600681a0 con 0x7fa9600ca350 2024-04-01T00:14:21.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.586+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fa948038020 con 0x7fa9600ca350 2024-04-01T00:14:21.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.586+0000 7fa95cff9640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 0x7fa94006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:21.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.586+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9480ac060 con 0x7fa9600ca350 2024-04-01T00:14:21.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.586+0000 7fa95f7fe640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 0x7fa94006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:21.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.587+0000 7fa95f7fe640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 0x7fa94006de50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9600b5ae0 tx=0x7fa950007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:21.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.591+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fa948031800 con 0x7fa9600ca350 2024-04-01T00:14:21.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.782+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa96002d2e0 con 0x7fa9600ca350 2024-04-01T00:14:21.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.783+0000 7fa95cff9640 1 -- 172.21.15.64:0/4018131927 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fa94807a620 con 0x7fa9600ca350 2024-04-01T00:14:21.783 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 msgr2=0x7fa94006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 0x7fa94006de50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9600b5ae0 tx=0x7fa950007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 msgr2=0x7fa9600c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098c0 tx=0x7fa9480046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 shutdown_connections 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.784+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa94006b990 0x7fa94006de50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa960145290 0x7fa9600cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9600cae40 0x7fa9601557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 --2- 172.21.15.64:0/4018131927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9600ca350 0x7fa9600c9560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 >> 172.21.15.64:0/4018131927 conn(0x7fa9600cfa90 msgr2=0x7fa96013a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:21.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 shutdown_connections 2024-04-01T00:14:21.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:21.785+0000 7fa9667e5640 1 -- 172.21.15.64:0/4018131927 wait complete. 2024-04-01T00:14:21.796 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-04-01T00:14:21.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:21.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:21.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:21.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:14:21.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:21.823 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-01T00:14:21.886 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:21.886 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:21.903 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:21.903 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:21.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.063+0000 7f0aacda1640 1 Processor -- start 2024-04-01T00:14:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.063+0000 7f0aacda1640 1 -- start start 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0aa806d560 0x7f0aa8071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa8071a80 0x7f0aa80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa80afe60 con 0x7f0aa8071a80 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa815bf50 con 0x7f0aa806d560 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aacda1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa815eb90 con 0x7f0aa815bb70 2024-04-01T00:14:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa8071a80 0x7f0aa80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa8071a80 0x7f0aa80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33396/0 (socket says 172.21.15.64:33396) 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 -- 172.21.15.64:0/3935842435 learned_addr learned my addr 172.21.15.64:0/3935842435 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa5d9b640 1 --2- 172.21.15.64:0/3935842435 >> v2:172.21.15.100:3300/0 conn(0x7f0aa806d560 0x7f0aa8071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa659c640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 -- 172.21.15.64:0/3935842435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 msgr2=0x7f0aa815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 -- 172.21.15.64:0/3935842435 >> v2:172.21.15.100:3300/0 conn(0x7f0aa806d560 msgr2=0x7f0aa8071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 --2- 172.21.15.64:0/3935842435 >> v2:172.21.15.100:3300/0 conn(0x7f0aa806d560 0x7f0aa8071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.064+0000 7f0aa6d9d640 1 -- 172.21.15.64:0/3935842435 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa815ee70 con 0x7f0aa8071a80 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.065+0000 7f0aa6d9d640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8071a80 0x7f0aa80c9cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8067bc0 tx=0x7f0a9802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c9cbf871b8ea82 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.065+0000 7f0aa559a640 1 -- 172.21.15.64:0/3935842435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9802ca70 con 0x7f0aa8071a80 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.065+0000 7f0aa559a640 1 -- 172.21.15.64:0/3935842435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9802cc10 con 0x7f0aa8071a80 2024-04-01T00:14:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.065+0000 7f0aa559a640 1 -- 172.21.15.64:0/3935842435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980343f0 con 0x7f0aa8071a80 2024-04-01T00:14:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 -- 172.21.15.64:0/3935842435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8071a80 msgr2=0x7f0aa80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8071a80 0x7f0aa80c9cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8067bc0 tx=0x7f0a9802bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 -- 172.21.15.64:0/3935842435 shutdown_connections 2024-04-01T00:14:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8071a80 0x7f0aa80c9cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 --2- 172.21.15.64:0/3935842435 >> v2:172.21.15.100:3300/0 conn(0x7f0aa806d560 0x7f0aa8071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 --2- 172.21.15.64:0/3935842435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 -- 172.21.15.64:0/3935842435 >> 172.21.15.64:0/3935842435 conn(0x7f0aa80d03e0 msgr2=0x7f0aa80d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 -- 172.21.15.64:0/3935842435 shutdown_connections 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.066+0000 7f0aacda1640 1 -- 172.21.15.64:0/3935842435 wait complete. 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.067+0000 7f0aacda1640 1 Processor -- start 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.067+0000 7f0aacda1640 1 -- start start 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 0x7f0aa8150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa806d9c0 con 0x7f0aa806d560 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa80c97d0 con 0x7f0aa8071a80 2024-04-01T00:14:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aacda1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa815f430 con 0x7f0aa815bb70 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33402/0 (socket says 172.21.15.64:33402) 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 -- 172.21.15.64:0/1971007459 learned_addr learned my addr 172.21.15.64:0/1971007459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa5d9b640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 0x7f0aa8150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa6d9d640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 -- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 msgr2=0x7f0aa815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 -- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 msgr2=0x7f0aa8150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 0x7f0aa8150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa659c640 1 -- 172.21.15.64:0/1971007459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa8160b20 con 0x7f0aa806d560 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.068+0000 7f0aa5d9b640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 0x7f0aa8150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0aa659c640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000e8a0 tx=0x7f0aa000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0017070 con 0x7f0aa806d560 2024-04-01T00:14:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa000c940 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa8160610 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000cd30 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.069+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81f7e50 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.070+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa80681a0 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.070+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0aa001a070 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.071+0000 7f0a9effd640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 0x7f0a8806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.071+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0aa008a610 con 0x7f0aa806d560 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.071+0000 7f0aa5d9b640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 0x7f0a8806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.071+0000 7f0aa5d9b640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 0x7f0a8806de80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80caa80 tx=0x7f0a90009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.075+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0aa0058bd0 con 0x7f0aa806d560 2024-04-01T00:14:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.257+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0aa806bfc0 con 0x7f0aa806d560 2024-04-01T00:14:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.258+0000 7f0a9effd640 1 -- 172.21.15.64:0/1971007459 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f0aa005cc70 con 0x7f0aa806d560 2024-04-01T00:14:22.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:22.259 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 msgr2=0x7f0a8806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 0x7f0a8806de80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80caa80 tx=0x7f0a90009190 comp rx=0 tx=0).stop 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 msgr2=0x7f0aa81600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000e8a0 tx=0x7f0aa000ed60 comp rx=0 tx=0).stop 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 shutdown_connections 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a8806b9c0 0x7f0a8806de80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0aa815bb70 0x7f0aa815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8071a80 0x7f0aa8150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 --2- 172.21.15.64:0/1971007459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa806d560 0x7f0aa81600d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 >> 172.21.15.64:0/1971007459 conn(0x7f0aa80d03e0 msgr2=0x7f0aa80cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 shutdown_connections 2024-04-01T00:14:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.260+0000 7f0aacda1640 1 -- 172.21.15.64:0/1971007459 wait complete. 2024-04-01T00:14:22.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:22.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 Processor -- start 2024-04-01T00:14:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 -- start start 2024-04-01T00:14:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f720c0cfe50 0x7f720c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c0cc730 con 0x7f720c0b1850 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c0cca10 con 0x7f720c0b0ba0 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.376+0000 7f7212af4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0b0f80 con 0x7f720c0cfe50 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f720bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f720c0b1850 0x7f720c0cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49832/0 (socket says 172.21.15.64:49832) 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f720bfff640 1 -- 172.21.15.64:0/1450136476 learned_addr learned my addr 172.21.15.64:0/1450136476 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f7211091640 1 --2- 172.21.15.64:0/1450136476 >> v2:172.21.15.100:3300/0 conn(0x7f720c0cfe50 0x7f720c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929854042 0 0) 0x7f720c0cc730 con 0x7f720c0b1850 2024-04-01T00:14:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71f4003520 con 0x7f720c0b1850 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f7211091640 1 -- 172.21.15.64:0/1450136476 >> v1:172.21.15.100:6789/0 conn(0x7f720c0b0ba0 legacy=0x7f720c0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432882312 0 0) 0x7f720c0cca10 con 0x7f720c0b0ba0 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f7211091640 1 -- 172.21.15.64:0/1450136476 >> v1:172.21.15.64:6789/0 conn(0x7f720c0b1850 legacy=0x7f720c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.377+0000 7f7211091640 1 -- 172.21.15.64:0/1450136476 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0cccf0 con 0x7f720c0cfe50 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f7211091640 1 --2- 172.21.15.64:0/1450136476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0cfe50 0x7f720c0cd7d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0097a0 tx=0x7f71fc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e4d13d9f39adaf server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc035020 con 0x7f720c0cfe50 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc031830 con 0x7f720c0cfe50 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f720b7fe640 1 -- 172.21.15.64:0/1450136476 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc031be0 con 0x7f720c0cfe50 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f7212af4640 1 -- 172.21.15.64:0/1450136476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0cfe50 msgr2=0x7f720c0cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.378+0000 7f7212af4640 1 --2- 172.21.15.64:0/1450136476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0cfe50 0x7f720c0cd7d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0097a0 tx=0x7f71fc02b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.379+0000 7f7212af4640 1 -- 172.21.15.64:0/1450136476 shutdown_connections 2024-04-01T00:14:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.379+0000 7f7212af4640 1 --2- 172.21.15.64:0/1450136476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0cfe50 0x7f720c0cd7d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.379+0000 7f7212af4640 1 -- 172.21.15.64:0/1450136476 >> 172.21.15.64:0/1450136476 conn(0x7f720c072810 msgr2=0x7f720c072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.379+0000 7f7212af4640 1 -- 172.21.15.64:0/1450136476 shutdown_connections 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.379+0000 7f7212af4640 1 -- 172.21.15.64:0/1450136476 wait complete. 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.380+0000 7f7212af4640 1 Processor -- start 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.380+0000 7f7212af4640 1 -- start start 2024-04-01T00:14:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 0x7f720c155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720c0cfe50 0x7f720c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f720c162e10 con 0x7f720c0b1850 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0cd2b0 con 0x7f720c0b0ba0 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f720c0caa30 con 0x7f720c0cfe50 2024-04-01T00:14:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33404/0 (socket says 172.21.15.64:33404) 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 -- 172.21.15.64:0/1568515854 learned_addr learned my addr 172.21.15.64:0/1568515854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7210890640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 0x7f720c155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7211091640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720c0cfe50 0x7f720c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 -- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720c0cfe50 msgr2=0x7f720c0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720c0cfe50 0x7f720c0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 -- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 msgr2=0x7f720c155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 0x7f720c155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 -- 172.21.15.64:0/1568515854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0afd70 con 0x7f720c0b1850 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7210890640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 0x7f720c155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f720bfff640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f720000c4d0 tx=0x7f720000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200017070 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7200005d00 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000072b0 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.381+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c160890 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.382+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c155c70 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.382+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c0681a0 con 0x7f720c0b1850 2024-04-01T00:14:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.383+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f720001a070 con 0x7f720c0b1850 2024-04-01T00:14:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.383+0000 7f72097fa640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 0x7f71f806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.383+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7200089b20 con 0x7f720c0b1850 2024-04-01T00:14:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.383+0000 7f7210890640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 0x7f71f806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.384+0000 7f7210890640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 0x7f71f806de50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72040012e0 tx=0x7f7204009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.388+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f72000580f0 con 0x7f720c0b1850 2024-04-01T00:14:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.570+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f720c0b2d00 con 0x7f720c0b1850 2024-04-01T00:14:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.570+0000 7f72097fa640 1 -- 172.21.15.64:0/1568515854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f720005c190 con 0x7f720c0b1850 2024-04-01T00:14:22.571 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 msgr2=0x7f71f806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 0x7f71f806de50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72040012e0 tx=0x7f7204009040 comp rx=0 tx=0).stop 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 msgr2=0x7f720c0c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f720000c4d0 tx=0x7f720000c990 comp rx=0 tx=0).stop 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 shutdown_connections 2024-04-01T00:14:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71f806b990 0x7f71f806de50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720c0cfe50 0x7f720c0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f720c0b1850 0x7f720c0c9680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 --2- 172.21.15.64:0/1568515854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f720c0b0ba0 0x7f720c155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 >> 172.21.15.64:0/1568515854 conn(0x7f720c072810 msgr2=0x7f720c072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 shutdown_connections 2024-04-01T00:14:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.572+0000 7f7212af4640 1 -- 172.21.15.64:0/1568515854 wait complete. 2024-04-01T00:14:22.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:22.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.688+0000 7fde87c29640 1 Processor -- start 2024-04-01T00:14:22.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.688+0000 7fde87c29640 1 -- start start 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.688+0000 7fde87c29640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde8015f570 0x7fde8015d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.688+0000 7fde87c29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde8015da40 0x7fde800b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.688+0000 7fde87c29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde800b26d0 0x7fde800d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde87c29640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fde800b1d00 con 0x7fde8015f570 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde87c29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fde800b13d0 con 0x7fde8015da40 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde87c29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fde800b16b0 con 0x7fde800b26d0 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde8015f570 0x7fde8015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde8015f570 0x7fde8015d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33418/0 (socket says 172.21.15.64:33418) 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 -- 172.21.15.64:0/1591369841 learned_addr learned my addr 172.21.15.64:0/1591369841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde861c6640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde800b26d0 0x7fde800d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde851c4640 1 --2- 172.21.15.64:0/1591369841 >> v2:172.21.15.100:3300/0 conn(0x7fde8015da40 0x7fde800b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 -- 172.21.15.64:0/1591369841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde800b26d0 msgr2=0x7fde800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde800b26d0 0x7fde800d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 -- 172.21.15.64:0/1591369841 >> v2:172.21.15.100:3300/0 conn(0x7fde8015da40 msgr2=0x7fde800b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 --2- 172.21.15.64:0/1591369841 >> v2:172.21.15.100:3300/0 conn(0x7fde8015da40 0x7fde800b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.689+0000 7fde859c5640 1 -- 172.21.15.64:0/1591369841 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde800b1990 con 0x7fde8015f570 2024-04-01T00:14:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde859c5640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde8015f570 0x7fde8015d500 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde80159f30 tx=0x7fde7402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8886d478f5b5ad90 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde849c3640 1 -- 172.21.15.64:0/1591369841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7402e070 con 0x7fde8015f570 2024-04-01T00:14:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde849c3640 1 -- 172.21.15.64:0/1591369841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde74037070 con 0x7fde8015f570 2024-04-01T00:14:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde849c3640 1 -- 172.21.15.64:0/1591369841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74031a00 con 0x7fde8015f570 2024-04-01T00:14:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 -- 172.21.15.64:0/1591369841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde8015f570 msgr2=0x7fde8015d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde8015f570 0x7fde8015d500 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde80159f30 tx=0x7fde7402b880 comp rx=0 tx=0).stop 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 -- 172.21.15.64:0/1591369841 shutdown_connections 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde800b26d0 0x7fde800d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 --2- 172.21.15.64:0/1591369841 >> v2:172.21.15.100:3300/0 conn(0x7fde8015da40 0x7fde800b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 --2- 172.21.15.64:0/1591369841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde8015f570 0x7fde8015d500 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.690+0000 7fde87c29640 1 -- 172.21.15.64:0/1591369841 >> 172.21.15.64:0/1591369841 conn(0x7fde8009a980 msgr2=0x7fde8009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.691+0000 7fde87c29640 1 -- 172.21.15.64:0/1591369841 shutdown_connections 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.691+0000 7fde87c29640 1 -- 172.21.15.64:0/1591369841 wait complete. 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 Processor -- start 2024-04-01T00:14:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 -- start start 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde8015da40 0x7fde800c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde8015f570 0x7fde800c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde800d21b0 con 0x7fde800b26d0 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde800d2030 con 0x7fde8015f570 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde87c29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fde800d2330 con 0x7fde8015da40 2024-04-01T00:14:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.692+0000 7fde859c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33430/0 (socket says 172.21.15.64:33430) 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 -- 172.21.15.64:0/4094460753 learned_addr learned my addr 172.21.15.64:0/4094460753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde861c6640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde8015f570 0x7fde800c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde851c4640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde8015da40 0x7fde800c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 -- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde8015da40 msgr2=0x7fde800c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde8015da40 0x7fde800c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 -- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde8015f570 msgr2=0x7fde800c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde8015f570 0x7fde800c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 -- 172.21.15.64:0/4094460753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde800c8d40 con 0x7fde800b26d0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde859c5640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde74000c00 tx=0x7fde740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74032670 con 0x7fde800b26d0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde74032810 con 0x7fde800b26d0 2024-04-01T00:14:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74032c00 con 0x7fde800b26d0 2024-04-01T00:14:22.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde800c8fe0 con 0x7fde800b26d0 2024-04-01T00:14:22.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.693+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde800c95a0 con 0x7fde800b26d0 2024-04-01T00:14:22.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.694+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde800681a0 con 0x7fde800b26d0 2024-04-01T00:14:22.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.695+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fde7404c060 con 0x7fde800b26d0 2024-04-01T00:14:22.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.695+0000 7fde727fc640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 0x7fde6406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:22.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.695+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fde740abfe0 con 0x7fde800b26d0 2024-04-01T00:14:22.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.695+0000 7fde851c4640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 0x7fde6406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:22.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.696+0000 7fde851c4640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 0x7fde6406de50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde80159f30 tx=0x7fde7c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.700+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fde74040070 con 0x7fde800b26d0 2024-04-01T00:14:22.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.882+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde800b1190 con 0x7fde800b26d0 2024-04-01T00:14:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.883+0000 7fde727fc640 1 -- 172.21.15.64:0/4094460753 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fde7407a660 con 0x7fde800b26d0 2024-04-01T00:14:22.884 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 msgr2=0x7fde6406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 0x7fde6406de50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde80159f30 tx=0x7fde7c009190 comp rx=0 tx=0).stop 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 msgr2=0x7fde80155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde74000c00 tx=0x7fde740046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 shutdown_connections 2024-04-01T00:14:22.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fde6406b990 0x7fde6406de50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde8015f570 0x7fde800c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde8015da40 0x7fde800c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 --2- 172.21.15.64:0/4094460753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde800b26d0 0x7fde80155c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 >> 172.21.15.64:0/4094460753 conn(0x7fde8009a980 msgr2=0x7fde8006f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 shutdown_connections 2024-04-01T00:14:22.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:22.885+0000 7fde87c29640 1 -- 172.21.15.64:0/4094460753 wait complete. 2024-04-01T00:14:22.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 Processor -- start 2024-04-01T00:14:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 -- start start 2024-04-01T00:14:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c161a50 0x7f1b0c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b2df0 0x7f1b0c0d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0c0b52c0 con 0x7f1b0c0b3940 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f1b0c0b5070 con 0x7f1b0c161a50 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b12ed4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c0d4bf0 con 0x7f1b0c0b2df0 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b10c70640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c161a50 0x7f1b0c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.001+0000 7f1b11471640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1b0c0b3940 0x7f1b0c0b3d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49834/0 (socket says 172.21.15.64:49834) 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b10c70640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c161a50 0x7f1b0c0d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33436/0 (socket says 172.21.15.64:33436) 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b11471640 1 -- 172.21.15.64:0/4247605700 learned_addr learned my addr 172.21.15.64:0/4247605700 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b0bfff640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b2df0 0x7f1b0c0d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b10c70640 1 -- 172.21.15.64:0/4247605700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b2df0 msgr2=0x7f1b0c0d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b10c70640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b2df0 0x7f1b0c0d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b10c70640 1 -- 172.21.15.64:0/4247605700 >> v1:172.21.15.64:6789/0 conn(0x7f1b0c0b3940 legacy=0x7f1b0c0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b0b7fe640 1 -- 172.21.15.64:0/4247605700 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997309309 0 0) 0x7f1b0c0b52c0 con 0x7f1b0c0b3940 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.002+0000 7f1b10c70640 1 -- 172.21.15.64:0/4247605700 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0b55a0 con 0x7f1b0c161a50 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b10c70640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c0d0810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b51f0 tx=0x7f1b0002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3ddd3d8818c9df server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b0b7fe640 1 -- 172.21.15.64:0/4247605700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00031770 con 0x7f1b0c161a50 2024-04-01T00:14:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b0b7fe640 1 -- 172.21.15.64:0/4247605700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b00037070 con 0x7f1b0c161a50 2024-04-01T00:14:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b0b7fe640 1 -- 172.21.15.64:0/4247605700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00031b30 con 0x7f1b0c161a50 2024-04-01T00:14:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 -- 172.21.15.64:0/4247605700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 msgr2=0x7f1b0c0d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c0d0810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b51f0 tx=0x7f1b0002b810 comp rx=0 tx=0).stop 2024-04-01T00:14:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 -- 172.21.15.64:0/4247605700 shutdown_connections 2024-04-01T00:14:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b2df0 0x7f1b0c0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/4247605700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c0d0810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 -- 172.21.15.64:0/4247605700 >> 172.21.15.64:0/4247605700 conn(0x7f1b0c09d1c0 msgr2=0x7f1b0c0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 -- 172.21.15.64:0/4247605700 shutdown_connections 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.003+0000 7f1b12ed4640 1 -- 172.21.15.64:0/4247605700 wait complete. 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.004+0000 7f1b12ed4640 1 Processor -- start 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 -- start start 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b0c0b2df0 0x7f1b0c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b3940 0x7f1b0c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c0cfaf0 con 0x7f1b0c161a50 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c0d4880 con 0x7f1b0c0b2df0 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.005+0000 7f1b12ed4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c0cfc70 con 0x7f1b0c0b3940 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33440/0 (socket says 172.21.15.64:33440) 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 -- 172.21.15.64:0/1732764876 learned_addr learned my addr 172.21.15.64:0/1732764876 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b10c70640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b0c0b2df0 0x7f1b0c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b0bfff640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b3940 0x7f1b0c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 -- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b3940 msgr2=0x7f1b0c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b3940 0x7f1b0c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 -- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b0c0b2df0 msgr2=0x7f1b0c0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b0c0b2df0 0x7f1b0c0b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 -- 172.21.15.64:0/1732764876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0c8910 con 0x7f1b0c161a50 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b11471640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7a0 tx=0x7f1b0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04018070 con 0x7f1b0c161a50 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0400a640 con 0x7f1b0c161a50 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c0c8bb0 con 0x7f1b0c161a50 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400aab0 con 0x7f1b0c161a50 2024-04-01T00:14:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.006+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c0c9170 con 0x7f1b0c161a50 2024-04-01T00:14:23.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.008+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c0681a0 con 0x7f1b0c161a50 2024-04-01T00:14:23.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.008+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1b0401b070 con 0x7f1b0c161a50 2024-04-01T00:14:23.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.008+0000 7f1b097fa640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 0x7f1af006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.008+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b04089d10 con 0x7f1b0c161a50 2024-04-01T00:14:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.009+0000 7f1b10c70640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 0x7f1af006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.009+0000 7f1b10c70640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 0x7f1af006de50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b00036000 tx=0x7f1b00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.013+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1b04058360 con 0x7f1b0c161a50 2024-04-01T00:14:23.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.193+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b0c06bbd0 con 0x7f1b0c161a50 2024-04-01T00:14:23.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.193+0000 7f1b097fa640 1 -- 172.21.15.64:0/1732764876 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1b0405c400 con 0x7f1b0c161a50 2024-04-01T00:14:23.194 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 msgr2=0x7f1af006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 0x7f1af006de50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b00036000 tx=0x7f1b00008030 comp rx=0 tx=0).stop 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 msgr2=0x7f1b0c156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7a0 tx=0x7f1b0400cc60 comp rx=0 tx=0).stop 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 shutdown_connections 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1af006b990 0x7f1af006de50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c161a50 0x7f1b0c156770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b0c0b3940 0x7f1b0c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 --2- 172.21.15.64:0/1732764876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b0c0b2df0 0x7f1b0c0b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 >> 172.21.15.64:0/1732764876 conn(0x7f1b0c09d1c0 msgr2=0x7f1b0c0d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 shutdown_connections 2024-04-01T00:14:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.195+0000 7f1b12ed4640 1 -- 172.21.15.64:0/1732764876 wait complete. 2024-04-01T00:14:23.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.309+0000 7f89bc3ac640 1 Processor -- start 2024-04-01T00:14:23.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.309+0000 7f89bc3ac640 1 -- start start 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89bc3ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f89b4144720 0x7f89b406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89bc3ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89b40cab60 0x7f89b406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89bc3ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b40cb020 con 0x7f89b406eb30 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89bc3ac640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f89b4144b00 con 0x7f89b40cab60 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89bc3ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f89b406ef60 con 0x7f89b4144720 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89b40cab60 0x7f89b406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89b40cab60 0x7f89b406e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33456/0 (socket says 172.21.15.64:33456) 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 -- 172.21.15.64:0/3274729515 learned_addr learned my addr 172.21.15.64:0/3274729515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89ba148640 1 --2- 172.21.15.64:0/3274729515 >> v2:172.21.15.100:3300/0 conn(0x7f89b4144720 0x7f89b406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 -- 172.21.15.64:0/3274729515 >> v2:172.21.15.100:3300/0 conn(0x7f89b4144720 msgr2=0x7f89b406ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:23.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 --2- 172.21.15.64:0/3274729515 >> v2:172.21.15.100:3300/0 conn(0x7f89b4144720 0x7f89b406ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 -- 172.21.15.64:0/3274729515 >> v1:172.21.15.100:6789/0 conn(0x7f89b406eb30 legacy=0x7f89b415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.310+0000 7f89b9947640 1 -- 172.21.15.64:0/3274729515 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40cb300 con 0x7f89b40cab60 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89b9947640 1 --2- 172.21.15.64:0/3274729515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b40cab60 0x7f89b406e5f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f89b40d0b90 tx=0x7f89a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72a843c93f7e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89b9146640 1 -- 172.21.15.64:0/3274729515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4034070 con 0x7f89b40cab60 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89b9146640 1 -- 172.21.15.64:0/3274729515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a4030860 con 0x7f89b40cab60 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89b9146640 1 -- 172.21.15.64:0/3274729515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4030c30 con 0x7f89b40cab60 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89bc3ac640 1 -- 172.21.15.64:0/3274729515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b40cab60 msgr2=0x7f89b406e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.311+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/3274729515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b40cab60 0x7f89b406e5f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f89b40d0b90 tx=0x7f89a402b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 -- 172.21.15.64:0/3274729515 shutdown_connections 2024-04-01T00:14:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/3274729515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b40cab60 0x7f89b406e5f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/3274729515 >> v2:172.21.15.100:3300/0 conn(0x7f89b4144720 0x7f89b406ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 -- 172.21.15.64:0/3274729515 >> 172.21.15.64:0/3274729515 conn(0x7f89b40cfa90 msgr2=0x7f89b40cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 -- 172.21.15.64:0/3274729515 shutdown_connections 2024-04-01T00:14:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.312+0000 7f89bc3ac640 1 -- 172.21.15.64:0/3274729515 wait complete. 2024-04-01T00:14:23.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.313+0000 7f89bc3ac640 1 Processor -- start 2024-04-01T00:14:23.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.313+0000 7f89bc3ac640 1 -- start start 2024-04-01T00:14:23.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406eb30 0x7f89b413ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89b40cab60 0x7f89b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40cbba0 con 0x7f89b4144720 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40cb8c0 con 0x7f89b406eb30 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40cd7f0 con 0x7f89b40cab60 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33462/0 (socket says 172.21.15.64:33462) 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 -- 172.21.15.64:0/2038133928 learned_addr learned my addr 172.21.15.64:0/2038133928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba148640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406eb30 0x7f89b413ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89b9947640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89b40cab60 0x7f89b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 -- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89b40cab60 msgr2=0x7f89b406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89b40cab60 0x7f89b406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 -- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406eb30 msgr2=0x7f89b413ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406eb30 0x7f89b413ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 -- 172.21.15.64:0/2038133928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40c9170 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89ba949640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c6b0 tx=0x7f89ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003b70 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac003d30 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b40c9450 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac005310 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.314+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b40b2930 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.315+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b40681a0 con 0x7f89b4144720 2024-04-01T00:14:23.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.316+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f89ac0054b0 con 0x7f89b4144720 2024-04-01T00:14:23.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.316+0000 7f89aaffd640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 0x7f899806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.316+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89ac08a570 con 0x7f89b4144720 2024-04-01T00:14:23.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.316+0000 7f89ba148640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 0x7f899806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.317+0000 7f89ba148640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 0x7f899806de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009b40 tx=0x7f89b0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.320+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f89ac058b30 con 0x7f89b4144720 2024-04-01T00:14:23.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.515+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f89b40c9ec0 con 0x7f89b4144720 2024-04-01T00:14:23.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.515+0000 7f89aaffd640 1 -- 172.21.15.64:0/2038133928 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f89ac05cbd0 con 0x7f89b4144720 2024-04-01T00:14:23.516 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 msgr2=0x7f899806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 0x7f899806de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009b40 tx=0x7f89b0009190 comp rx=0 tx=0).stop 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 msgr2=0x7f89b406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c6b0 tx=0x7f89ac00cb70 comp rx=0 tx=0).stop 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 shutdown_connections 2024-04-01T00:14:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f899806b990 0x7f899806de50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4144720 0x7f89b406da20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89b40cab60 0x7f89b406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 --2- 172.21.15.64:0/2038133928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b406eb30 0x7f89b413ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 >> 172.21.15.64:0/2038133928 conn(0x7f89b40cfa90 msgr2=0x7f89b4072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 shutdown_connections 2024-04-01T00:14:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.517+0000 7f89bc3ac640 1 -- 172.21.15.64:0/2038133928 wait complete. 2024-04-01T00:14:23.529 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:23.529 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:23.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:23.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.645+0000 7f6971a55640 1 Processor -- start 2024-04-01T00:14:23.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.645+0000 7f6971a55640 1 -- start start 2024-04-01T00:14:23.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6971a55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6971a55640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f696c0cfae0 0x7f696c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6971a55640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696c0b3c80 con 0x7f696c0b3820 2024-04-01T00:14:23.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6971a55640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0d2ba0 con 0x7f696c0cfae0 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6971a55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0d2e80 con 0x7f696c1619e0 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f696c0cfae0 0x7f696c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f696c0cfae0 0x7f696c1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46810/0 (socket says 172.21.15.64:46810) 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 -- 172.21.15.64:0/2976673881 learned_addr learned my addr 172.21.15.64:0/2976673881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696a7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f696c0b3820 0x7f696c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51808/0 (socket says 172.21.15.64:51808) 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696affd640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 -- 172.21.15.64:0/2976673881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 msgr2=0x7f696c0d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f6969ffb640 1 -- 172.21.15.64:0/2976673881 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429472459 0 0) 0x7f696c0b3c80 con 0x7f696c0b3820 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 -- 172.21.15.64:0/2976673881 >> v1:172.21.15.100:6789/0 conn(0x7f696c0b3820 legacy=0x7f696c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.646+0000 7f696b7fe640 1 -- 172.21.15.64:0/2976673881 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c0ceeb0 con 0x7f696c0cfae0 2024-04-01T00:14:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.647+0000 7f696b7fe640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1620a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f695c009e50 tx=0x7f695c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7542d50018525286 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.647+0000 7f6969ffb640 1 -- 172.21.15.64:0/2976673881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c02ca70 con 0x7f696c0cfae0 2024-04-01T00:14:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.647+0000 7f6969ffb640 1 -- 172.21.15.64:0/2976673881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c02cc10 con 0x7f696c0cfae0 2024-04-01T00:14:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.647+0000 7f6969ffb640 1 -- 172.21.15.64:0/2976673881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0343f0 con 0x7f696c0cfae0 2024-04-01T00:14:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 -- 172.21.15.64:0/2976673881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 msgr2=0x7f696c1620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1620a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f695c009e50 tx=0x7f695c02bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 -- 172.21.15.64:0/2976673881 shutdown_connections 2024-04-01T00:14:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1620a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 --2- 172.21.15.64:0/2976673881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 -- 172.21.15.64:0/2976673881 >> 172.21.15.64:0/2976673881 conn(0x7f696c09d290 msgr2=0x7f696c0d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 -- 172.21.15.64:0/2976673881 shutdown_connections 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.648+0000 7f6971a55640 1 -- 172.21.15.64:0/2976673881 wait complete. 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.649+0000 7f6971a55640 1 Processor -- start 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.649+0000 7f6971a55640 1 -- start start 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f696c0b3820 0x7f696c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0b31b0 con 0x7f696c0cfae0 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0d2890 con 0x7f696c0b3820 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0ceb10 con 0x7f696c1619e0 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f696c0b3820 0x7f696c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46826/0 (socket says 172.21.15.64:46826) 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 -- 172.21.15.64:0/1884851591 learned_addr learned my addr 172.21.15.64:0/1884851591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696b7fe640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 -- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 msgr2=0x7f696c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 -- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f696c0b3820 msgr2=0x7f696c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f696c0b3820 0x7f696c0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 -- 172.21.15.64:0/1884851591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c1f40e0 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f696a7fc640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f695400c4d0 tx=0x7f695400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954018070 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954007500 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c0cc100 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.650+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954004a20 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.651+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c0cbb40 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.651+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c0681a0 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.652+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f695401b070 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.652+0000 7f6963fff640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 0x7f694c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.652+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f69540899e0 con 0x7f696c0cfae0 2024-04-01T00:14:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.652+0000 7f696affd640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 0x7f694c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.653+0000 7f696affd640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 0x7f694c06de80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f696c0ccc20 tx=0x7f6964009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.657+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6954058030 con 0x7f696c0cfae0 2024-04-01T00:14:23.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.838+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f696c09d730 con 0x7f696c0cfae0 2024-04-01T00:14:23.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.839+0000 7f6963fff640 1 -- 172.21.15.64:0/1884851591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f695405c0d0 con 0x7f696c0cfae0 2024-04-01T00:14:23.841 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:23.841 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 msgr2=0x7f694c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 0x7f694c06de80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f696c0ccc20 tx=0x7f6964009190 comp rx=0 tx=0).stop 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 msgr2=0x7f696c1f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f695400c4d0 tx=0x7f695400c990 comp rx=0 tx=0).stop 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 shutdown_connections 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f694c06b9c0 0x7f694c06de80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f696c1619e0 0x7f696c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f696c0cfae0 0x7f696c1f3ba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 --2- 172.21.15.64:0/1884851591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f696c0b3820 0x7f696c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 >> 172.21.15.64:0/1884851591 conn(0x7f696c09d290 msgr2=0x7f696c0b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 shutdown_connections 2024-04-01T00:14:23.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.842+0000 7f6971a55640 1 -- 172.21.15.64:0/1884851591 wait complete. 2024-04-01T00:14:23.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:23.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:23.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.958+0000 7f74e1207640 1 Processor -- start 2024-04-01T00:14:23.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.958+0000 7f74e1207640 1 -- start start 2024-04-01T00:14:23.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74e1207640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f74dc1459a0 0x7f74dc0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74e1207640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74e1207640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74dc0cf9f0 con 0x7f74dc06b600 2024-04-01T00:14:23.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74e1207640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f74dc0cfcd0 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74e1207640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f74dc0cffb0 con 0x7f74dc1459a0 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74db59e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f74dc06b600 0x7f74dc071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42382/0 (socket says 172.21.15.64:42382) 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74db59e640 1 -- 172.21.15.64:0/1481299256 learned_addr learned my addr 172.21.15.64:0/1481299256 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46838/0 (socket says 172.21.15.64:46838) 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74dad9d640 1 --2- 172.21.15.64:0/1481299256 >> v2:172.21.15.100:3300/0 conn(0x7f74dc1459a0 0x7f74dc0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 -- 172.21.15.64:0/1481299256 >> v2:172.21.15.100:3300/0 conn(0x7f74dc1459a0 msgr2=0x7f74dc0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 --2- 172.21.15.64:0/1481299256 >> v2:172.21.15.100:3300/0 conn(0x7f74dc1459a0 0x7f74dc0c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 -- 172.21.15.64:0/1481299256 >> v1:172.21.15.64:6789/0 conn(0x7f74dc06b600 legacy=0x7f74dc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74d9d9b640 1 -- 172.21.15.64:0/1481299256 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 776473035 0 0) 0x7f74dc0cf9f0 con 0x7f74dc06b600 2024-04-01T00:14:23.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.959+0000 7f74da59c640 1 -- 172.21.15.64:0/1481299256 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc0d0290 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.960+0000 7f74da59c640 1 --2- 172.21.15.64:0/1481299256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74dc073160 tx=0x7f74c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512a08ce78a9910b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.960+0000 7f74d9d9b640 1 -- 172.21.15.64:0/1481299256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4035020 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.960+0000 7f74d9d9b640 1 -- 172.21.15.64:0/1481299256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4031830 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.960+0000 7f74d9d9b640 1 -- 172.21.15.64:0/1481299256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4031c00 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 -- 172.21.15.64:0/1481299256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc0cf4c0 msgr2=0x7f74dc06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 --2- 172.21.15.64:0/1481299256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74dc073160 tx=0x7f74c402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 -- 172.21.15.64:0/1481299256 shutdown_connections 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 --2- 172.21.15.64:0/1481299256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc06b0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 --2- 172.21.15.64:0/1481299256 >> v2:172.21.15.100:3300/0 conn(0x7f74dc1459a0 0x7f74dc0c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 -- 172.21.15.64:0/1481299256 >> 172.21.15.64:0/1481299256 conn(0x7f74dc0d0bb0 msgr2=0x7f74dc0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 -- 172.21.15.64:0/1481299256 shutdown_connections 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.961+0000 7f74e1207640 1 -- 172.21.15.64:0/1481299256 wait complete. 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.962+0000 7f74e1207640 1 Processor -- start 2024-04-01T00:14:23.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.962+0000 7f74e1207640 1 -- start start 2024-04-01T00:14:23.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc1f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74dc1459a0 0x7f74dc0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0c65a0 con 0x7f74dc06b600 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0c6420 con 0x7f74dc0cf4c0 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74e1207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc06ed30 con 0x7f74dc1459a0 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46842/0 (socket says 172.21.15.64:46842) 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 -- 172.21.15.64:0/3858373376 learned_addr learned my addr 172.21.15.64:0/3858373376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74da59c640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc1f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74db59e640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74dc1459a0 0x7f74dc0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 -- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74dc1459a0 msgr2=0x7f74dc0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74dc1459a0 0x7f74dc0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 -- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74dc0cf4c0 msgr2=0x7f74dc1f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:23.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc1f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.963+0000 7f74dad9d640 1 -- 172.21.15.64:0/3858373376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc0c9340 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74dad9d640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e870 tx=0x7f74d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4017070 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d400c940 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc141710 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d400cd30 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.964+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc0c9ba0 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.965+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc0681a0 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.965+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f74d401a070 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.966+0000 7f74d37fe640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 0x7f74bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.966+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74d408a600 con 0x7f74dc06b600 2024-04-01T00:14:23.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.966+0000 7f74da59c640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 0x7f74bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:23.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.966+0000 7f74da59c640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 0x7f74bc06de50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74c4036000 tx=0x7f74c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:23.970+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f74d4058c50 con 0x7f74dc06b600 2024-04-01T00:14:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.152+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74dc0ce300 con 0x7f74dc06b600 2024-04-01T00:14:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.152+0000 7f74d37fe640 1 -- 172.21.15.64:0/3858373376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f74d405ccf0 con 0x7f74dc06b600 2024-04-01T00:14:24.153 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.154+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 msgr2=0x7f74bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.154+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 0x7f74bc06de50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74c4036000 tx=0x7f74c4008030 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 msgr2=0x7f74dc1411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e870 tx=0x7f74d400ec70 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 shutdown_connections 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f74bc06b990 0x7f74bc06de50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74dc1459a0 0x7f74dc0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74dc0cf4c0 0x7f74dc1f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 --2- 172.21.15.64:0/3858373376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74dc06b600 0x7f74dc1411d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 >> 172.21.15.64:0/3858373376 conn(0x7f74dc0d0bb0 msgr2=0x7f74dc1429b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 shutdown_connections 2024-04-01T00:14:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.155+0000 7f74e1207640 1 -- 172.21.15.64:0/3858373376 wait complete. 2024-04-01T00:14:24.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:24.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 Processor -- start 2024-04-01T00:14:24.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 -- start start 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f744815e120 0x7f744806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f74480cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74480b0570 con 0x7f744806b600 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7448072d10 con 0x7f744815e120 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.270+0000 7f744d00f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74480c6040 con 0x7f74480726c0 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f744815e120 0x7f744806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f744815e120 0x7f744806b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:35508/0 (socket says 172.21.15.64:35508) 2024-04-01T00:14:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f744659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f744806b600 0x7f7448071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51818/0 (socket says 172.21.15.64:51818) 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 -- 172.21.15.64:0/1191049946 learned_addr learned my addr 172.21.15.64:0/1191049946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f744759e640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f74480cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 -- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 msgr2=0x7f74480cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f74480cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 -- 172.21.15.64:0/1191049946 >> v1:172.21.15.100:6789/0 conn(0x7f744806b600 legacy=0x7f7448071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7445d9b640 1 -- 172.21.15.64:0/1191049946 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504184296 0 0) 0x7f74480b0570 con 0x7f744806b600 2024-04-01T00:14:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.271+0000 7f7446d9d640 1 -- 172.21.15.64:0/1191049946 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74480c6320 con 0x7f744815e120 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f7446d9d640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744815e120 0x7f744806b0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7440009870 tx=0x7f744002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6bafbf1214a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f7445d9b640 1 -- 172.21.15.64:0/1191049946 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440034070 con 0x7f744815e120 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f7445d9b640 1 -- 172.21.15.64:0/1191049946 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440030840 con 0x7f744815e120 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f7445d9b640 1 -- 172.21.15.64:0/1191049946 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440030c10 con 0x7f744815e120 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 -- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744815e120 msgr2=0x7f744806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744815e120 0x7f744806b0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7440009870 tx=0x7f744002b980 comp rx=0 tx=0).stop 2024-04-01T00:14:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 -- 172.21.15.64:0/1191049946 shutdown_connections 2024-04-01T00:14:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f74480cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 --2- 172.21.15.64:0/1191049946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744815e120 0x7f744806b0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 -- 172.21.15.64:0/1191049946 >> 172.21.15.64:0/1191049946 conn(0x7f74480d0ba0 msgr2=0x7f74480d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 -- 172.21.15.64:0/1191049946 shutdown_connections 2024-04-01T00:14:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.272+0000 7f744d00f640 1 -- 172.21.15.64:0/1191049946 wait complete. 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.274+0000 7f744d00f640 1 Processor -- start 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.274+0000 7f744d00f640 1 -- start start 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.274+0000 7f744d00f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.274+0000 7f744d00f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f744814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.274+0000 7f744d00f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744d00f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f744815e500 con 0x7f744815e120 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744d00f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74480c68e0 con 0x7f744806b600 2024-04-01T00:14:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744d00f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74480c6e00 con 0x7f74480726c0 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f7446d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46856/0 (socket says 172.21.15.64:46856) 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f7446d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:35520/0 (socket says 172.21.15.64:35520) 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 -- 172.21.15.64:0/3477275140 learned_addr learned my addr 172.21.15.64:0/3477275140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744659c640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f744814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 -- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 msgr2=0x7f744814b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f744814b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 -- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 msgr2=0x7f744814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 -- 172.21.15.64:0/3477275140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744815b540 con 0x7f744815e120 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f7446d9d640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744759e640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f743800efc0 tx=0x7f743800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438018020 con 0x7f744815e120 2024-04-01T00:14:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438014480 con 0x7f744815e120 2024-04-01T00:14:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744815eaf0 con 0x7f744815e120 2024-04-01T00:14:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438014870 con 0x7f744815e120 2024-04-01T00:14:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.275+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744815b260 con 0x7f744815e120 2024-04-01T00:14:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.276+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74480681a0 con 0x7f744815e120 2024-04-01T00:14:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.277+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7438005d10 con 0x7f744815e120 2024-04-01T00:14:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.277+0000 7f742f7fe640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 0x7f743406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.277+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f743808b1d0 con 0x7f744815e120 2024-04-01T00:14:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.277+0000 7f7446d9d640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 0x7f743406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.278+0000 7f7446d9d640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 0x7f743406de50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7440036000 tx=0x7f7440008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.282+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f7438022070 con 0x7f744815e120 2024-04-01T00:14:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.463+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f74480cd3e0 con 0x7f744815e120 2024-04-01T00:14:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.464+0000 7f742f7fe640 1 -- 172.21.15.64:0/3477275140 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7438010040 con 0x7f744815e120 2024-04-01T00:14:24.465 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 msgr2=0x7f743406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 0x7f743406de50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7440036000 tx=0x7f7440008030 comp rx=0 tx=0).stop 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 msgr2=0x7f744815a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f743800efc0 tx=0x7f743800c490 comp rx=0 tx=0).stop 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 shutdown_connections 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f743406b990 0x7f743406de50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744815e120 0x7f744815a0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74480726c0 0x7f744814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 --2- 172.21.15.64:0/3477275140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f744806b600 0x7f744814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 >> 172.21.15.64:0/3477275140 conn(0x7f74480d0ba0 msgr2=0x7f74480aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 shutdown_connections 2024-04-01T00:14:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.466+0000 7f744d00f640 1 -- 172.21.15.64:0/3477275140 wait complete. 2024-04-01T00:14:24.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:24.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.581+0000 7fe5dede9640 1 Processor -- start 2024-04-01T00:14:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 -- start start 2024-04-01T00:14:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d8161a50 0x7fe5d80d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5d80b52c0 con 0x7fe5d80b2df0 2024-04-01T00:14:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe5d80b5070 con 0x7fe5d80d0d50 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dede9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d80d4bf0 con 0x7fe5d8161a50 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dd386640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dd386640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46866/0 (socket says 172.21.15.64:46866) 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dd386640 1 -- 172.21.15.64:0/1737337262 learned_addr learned my addr 172.21.15.64:0/1737337262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.582+0000 7fe5dcb85640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d8161a50 0x7fe5d80d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.583+0000 7fe5dd386640 1 -- 172.21.15.64:0/1737337262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d8161a50 msgr2=0x7fe5d80d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.583+0000 7fe5dd386640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d8161a50 0x7fe5d80d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.583+0000 7fe5dd386640 1 -- 172.21.15.64:0/1737337262 >> v1:172.21.15.100:6789/0 conn(0x7fe5d80b2df0 legacy=0x7fe5d80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.583+0000 7fe5dd386640 1 -- 172.21.15.64:0/1737337262 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d80b55a0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.583+0000 7fe5d77fe640 1 -- 172.21.15.64:0/1737337262 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3803466581 0 0) 0x7fe5d80b52c0 con 0x7fe5d80b2df0 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dd386640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b51f0 tx=0x7fe5c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87063cd2b8efa264 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5d77fe640 1 -- 172.21.15.64:0/1737337262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8035020 con 0x7fe5d80d0d50 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5d77fe640 1 -- 172.21.15.64:0/1737337262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c8031640 con 0x7fe5d80d0d50 2024-04-01T00:14:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5d77fe640 1 -- 172.21.15.64:0/1737337262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802d040 con 0x7fe5d80d0d50 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 -- 172.21.15.64:0/1737337262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 msgr2=0x7fe5d80cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b51f0 tx=0x7fe5c802b8f0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 -- 172.21.15.64:0/1737337262 shutdown_connections 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80cfaf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 --2- 172.21.15.64:0/1737337262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d8161a50 0x7fe5d80d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 -- 172.21.15.64:0/1737337262 >> 172.21.15.64:0/1737337262 conn(0x7fe5d809d1c0 msgr2=0x7fe5d80d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 -- 172.21.15.64:0/1737337262 shutdown_connections 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.584+0000 7fe5dede9640 1 -- 172.21.15.64:0/1737337262 wait complete. 2024-04-01T00:14:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.585+0000 7fe5dede9640 1 Processor -- start 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.585+0000 7fe5dede9640 1 -- start start 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d80b2df0 0x7fe5d815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d8161a50 0x7fe5d806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d80d4880 con 0x7fe5d80d0d50 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d80d1180 con 0x7fe5d8161a50 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dede9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d80d0030 con 0x7fe5d80b2df0 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46872/0 (socket says 172.21.15.64:46872) 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5d7fff640 1 -- 172.21.15.64:0/651306694 learned_addr learned my addr 172.21.15.64:0/651306694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dcb85640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d80b2df0 0x7fe5d815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.586+0000 7fe5dd386640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d8161a50 0x7fe5d806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 -- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d80b2df0 msgr2=0x7fe5d815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d80b2df0 0x7fe5d815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 -- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d8161a50 msgr2=0x7fe5d806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d8161a50 0x7fe5d806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 -- 172.21.15.64:0/651306694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d80cd1b0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d7fff640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c4e0 tx=0x7fe5cc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc017070 con 0x7fe5d80d0d50 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5cc0035f0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d80cc110 con 0x7fe5d80d0d50 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0039e0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.587+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d80cc6d0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.588+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d80681a0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.588+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe5cc01a070 con 0x7fe5d80d0d50 2024-04-01T00:14:24.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.589+0000 7fe5d57fa640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 0x7fe5c406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.589+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe5cc089a80 con 0x7fe5d80d0d50 2024-04-01T00:14:24.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.589+0000 7fe5dcb85640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 0x7fe5c406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.589+0000 7fe5dcb85640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 0x7fe5c406de50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80cf8f0 tx=0x7fe5d000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.593+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe5d80cd1b0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.775+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe5d80cab60 con 0x7fe5d80d0d50 2024-04-01T00:14:24.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.775+0000 7fe5d57fa640 1 -- 172.21.15.64:0/651306694 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe5cc0580d0 con 0x7fe5d80d0d50 2024-04-01T00:14:24.776 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 msgr2=0x7fe5c406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 0x7fe5c406de50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80cf8f0 tx=0x7fe5d000b040 comp rx=0 tx=0).stop 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 msgr2=0x7fe5d80ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c4e0 tx=0x7fe5cc00c9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 shutdown_connections 2024-04-01T00:14:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe5c406b990 0x7fe5c406de50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d8161a50 0x7fe5d806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d80d0d50 0x7fe5d80ca1b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 --2- 172.21.15.64:0/651306694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5d80b2df0 0x7fe5d815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 >> 172.21.15.64:0/651306694 conn(0x7fe5d809d1c0 msgr2=0x7fe5d80d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 shutdown_connections 2024-04-01T00:14:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.777+0000 7fe5dede9640 1 -- 172.21.15.64:0/651306694 wait complete. 2024-04-01T00:14:24.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.892+0000 7f34d2b01640 1 Processor -- start 2024-04-01T00:14:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.892+0000 7f34d2b01640 1 -- start start 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc144720 0x7f34cc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f34cc0cab60 0x7f34cc06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34cc06eb30 0x7f34cc15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc144b00 con 0x7f34cc06eb30 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc0cb020 con 0x7f34cc0cab60 2024-04-01T00:14:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d2b01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0cb300 con 0x7f34cc144720 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34cc06eb30 0x7f34cc15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34cc06eb30 0x7f34cc15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46886/0 (socket says 172.21.15.64:46886) 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 -- 172.21.15.64:0/1447894088 learned_addr learned my addr 172.21.15.64:0/1447894088 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34cbfff640 1 --2- 172.21.15.64:0/1447894088 >> v2:172.21.15.100:3300/0 conn(0x7f34cc0cab60 0x7f34cc06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d089d640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc144720 0x7f34cc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 -- 172.21.15.64:0/1447894088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc144720 msgr2=0x7f34cc06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc144720 0x7f34cc06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 -- 172.21.15.64:0/1447894088 >> v2:172.21.15.100:3300/0 conn(0x7f34cc0cab60 msgr2=0x7f34cc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 --2- 172.21.15.64:0/1447894088 >> v2:172.21.15.100:3300/0 conn(0x7f34cc0cab60 0x7f34cc06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.893+0000 7f34d109e640 1 -- 172.21.15.64:0/1447894088 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0cb5e0 con 0x7f34cc06eb30 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.894+0000 7f34d109e640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15e8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f34cc070990 tx=0x7f34c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11178a19fc798cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.894+0000 7f34cb7fe640 1 -- 172.21.15.64:0/1447894088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c402ca70 con 0x7f34cc06eb30 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.894+0000 7f34cb7fe640 1 -- 172.21.15.64:0/1447894088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c402cc10 con 0x7f34cc06eb30 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.894+0000 7f34cb7fe640 1 -- 172.21.15.64:0/1447894088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c40343f0 con 0x7f34cc06eb30 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 -- 172.21.15.64:0/1447894088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 msgr2=0x7f34cc15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15e8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f34cc070990 tx=0x7f34c402bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 -- 172.21.15.64:0/1447894088 shutdown_connections 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15e8f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 --2- 172.21.15.64:0/1447894088 >> v2:172.21.15.100:3300/0 conn(0x7f34cc0cab60 0x7f34cc06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 --2- 172.21.15.64:0/1447894088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc144720 0x7f34cc06ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 -- 172.21.15.64:0/1447894088 >> 172.21.15.64:0/1447894088 conn(0x7f34cc0cfa90 msgr2=0x7f34cc0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 -- 172.21.15.64:0/1447894088 shutdown_connections 2024-04-01T00:14:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.895+0000 7f34d2b01640 1 -- 172.21.15.64:0/1447894088 wait complete. 2024-04-01T00:14:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.896+0000 7f34d2b01640 1 Processor -- start 2024-04-01T00:14:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.896+0000 7f34d2b01640 1 -- start start 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 0x7f34cc15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 0x7f34cc15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0cd7f0 con 0x7f34cc06eb30 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0cbba0 con 0x7f34cc144720 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d2b01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0cd970 con 0x7f34cc0cab60 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46894/0 (socket says 172.21.15.64:46894) 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 -- 172.21.15.64:0/3437493888 learned_addr learned my addr 172.21.15.64:0/3437493888 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d109e640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 0x7f34cc15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34cbfff640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 0x7f34cc15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 -- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 msgr2=0x7f34cc15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 0x7f34cc15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 -- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 msgr2=0x7f34cc15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 0x7f34cc15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 -- 172.21.15.64:0/3437493888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc141c60 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34cbfff640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 0x7f34cc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d109e640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 0x7f34cc15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34d089d640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e870 tx=0x7f34c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.897+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0017070 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.898+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c000c940 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.898+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc15cff0 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.898+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000cd30 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.898+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc15d5b0 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.899+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc0681a0 con 0x7f34cc06eb30 2024-04-01T00:14:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.899+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f34c001a070 con 0x7f34cc06eb30 2024-04-01T00:14:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.899+0000 7f34c97fa640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 0x7f34b006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.899+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34c008aa10 con 0x7f34cc06eb30 2024-04-01T00:14:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.900+0000 7f34cbfff640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 0x7f34b006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:24.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.900+0000 7f34cbfff640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 0x7f34b006de50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f34cc137ec0 tx=0x7f34b8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:24.904+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f34c0058fd0 con 0x7f34cc06eb30 2024-04-01T00:14:25.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.097+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34cc143f10 con 0x7f34cc06eb30 2024-04-01T00:14:25.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.097+0000 7f34c97fa640 1 -- 172.21.15.64:0/3437493888 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f34c005d070 con 0x7f34cc06eb30 2024-04-01T00:14:25.098 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:25.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 msgr2=0x7f34b006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 0x7f34b006de50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f34cc137ec0 tx=0x7f34b8009190 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 msgr2=0x7f34cc15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e870 tx=0x7f34c000ed30 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 shutdown_connections 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f34b006b990 0x7f34b006de50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34cc144720 0x7f34cc15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34cc0cab60 0x7f34cc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 --2- 172.21.15.64:0/3437493888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34cc06eb30 0x7f34cc15a5a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 >> 172.21.15.64:0/3437493888 conn(0x7f34cc0cfa90 msgr2=0x7f34cc0b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 shutdown_connections 2024-04-01T00:14:25.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.099+0000 7f34d2b01640 1 -- 172.21.15.64:0/3437493888 wait complete. 2024-04-01T00:14:25.111 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-01T00:14:25.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:25.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-01T00:14:25.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:25.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:14:25.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:25.139 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-01T00:14:25.206 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:25.206 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:25.222 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:25.222 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:25.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 Processor -- start 2024-04-01T00:14:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 -- start start 2024-04-01T00:14:25.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07100723d0 0x7f07100b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0710070eb0 0x7f07100cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f071015fde0 con 0x7f07100d0480 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f07101600c0 con 0x7f0710070eb0 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.419+0000 7f0715981640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07101603a0 con 0x7f07100723d0 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070e7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0710070eb0 0x7f07100cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f07100d0480 0x7f07100cc620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42386/0 (socket says 172.21.15.64:42386) 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070f7fe640 1 -- 172.21.15.64:0/2934745055 learned_addr learned my addr 172.21.15.64:0/2934745055 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070effd640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07100723d0 0x7f07100b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070e7fc640 1 -- 172.21.15.64:0/2934745055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07100723d0 msgr2=0x7f07100b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070e7fc640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07100723d0 0x7f07100b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070e7fc640 1 -- 172.21.15.64:0/2934745055 >> v1:172.21.15.64:6789/0 conn(0x7f07100d0480 legacy=0x7f07100cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070dffb640 1 -- 172.21.15.64:0/2934745055 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713041132 0 0) 0x7f071015fde0 con 0x7f07100d0480 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.420+0000 7f070e7fc640 1 -- 172.21.15.64:0/2934745055 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0710160680 con 0x7f0710070eb0 2024-04-01T00:14:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f070e7fc640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710070eb0 0x7f07100cff40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06f80097a0 tx=0x7f06f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfa01f659477e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f070dffb640 1 -- 172.21.15.64:0/2934745055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f80315a0 con 0x7f0710070eb0 2024-04-01T00:14:25.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f070dffb640 1 -- 172.21.15.64:0/2934745055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8037070 con 0x7f0710070eb0 2024-04-01T00:14:25.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f070dffb640 1 -- 172.21.15.64:0/2934745055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8031960 con 0x7f0710070eb0 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 -- 172.21.15.64:0/2934745055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710070eb0 msgr2=0x7f07100cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710070eb0 0x7f07100cff40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06f80097a0 tx=0x7f06f802b840 comp rx=0 tx=0).stop 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 -- 172.21.15.64:0/2934745055 shutdown_connections 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710070eb0 0x7f07100cff40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 --2- 172.21.15.64:0/2934745055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07100723d0 0x7f07100b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 -- 172.21.15.64:0/2934745055 >> 172.21.15.64:0/2934745055 conn(0x7f071006b4e0 msgr2=0x7f071006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 -- 172.21.15.64:0/2934745055 shutdown_connections 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.421+0000 7f0715981640 1 -- 172.21.15.64:0/2934745055 wait complete. 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 Processor -- start 2024-04-01T00:14:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 -- start start 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0710070eb0 0x7f0710150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07100d0480 0x7f071015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07100718a0 con 0x7f07100723d0 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0710071720 con 0x7f07100d0480 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.423+0000 7f0715981640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07100727b0 con 0x7f0710070eb0 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46904/0 (socket says 172.21.15.64:46904) 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 -- 172.21.15.64:0/568591819 learned_addr learned my addr 172.21.15.64:0/568591819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070f7fe640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07100d0480 0x7f071015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070effd640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0710070eb0 0x7f0710150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 -- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0710070eb0 msgr2=0x7f0710150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0710070eb0 0x7f0710150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 -- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07100d0480 msgr2=0x7f071015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07100d0480 0x7f071015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 -- 172.21.15.64:0/568591819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071015cc70 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f070e7fc640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f06f8034040 tx=0x7f06f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8032630 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f80327f0 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071015cf50 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8032be0 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.424+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071015d510 con 0x7f07100723d0 2024-04-01T00:14:25.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.425+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07100681a0 con 0x7f07100723d0 2024-04-01T00:14:25.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.426+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f06f804e430 con 0x7f07100723d0 2024-04-01T00:14:25.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.426+0000 7f0707fff640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 0x7f06f006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.426+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06f802f030 con 0x7f07100723d0 2024-04-01T00:14:25.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.426+0000 7f070effd640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 0x7f06f006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.427+0000 7f070effd640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 0x7f06f006de50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0708009b40 tx=0x7f0708009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.430+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f06f8040070 con 0x7f07100723d0 2024-04-01T00:14:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.613+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07100ce250 con 0x7f07100723d0 2024-04-01T00:14:25.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.614+0000 7f0707fff640 1 -- 172.21.15.64:0/568591819 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f06f807abb0 con 0x7f07100723d0 2024-04-01T00:14:25.616 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:25.616 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:13:48.022599+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 msgr2=0x7f06f006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 0x7f06f006de50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0708009b40 tx=0x7f0708009190 comp rx=0 tx=0).stop 2024-04-01T00:14:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 msgr2=0x7f071009c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f06f8034040 tx=0x7f06f80046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 shutdown_connections 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06f006b990 0x7f06f006de50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07100d0480 0x7f071015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100723d0 0x7f071009c9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 --2- 172.21.15.64:0/568591819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0710070eb0 0x7f0710150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 >> 172.21.15.64:0/568591819 conn(0x7f071006b4e0 msgr2=0x7f07100b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 shutdown_connections 2024-04-01T00:14:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.616+0000 7f0715981640 1 -- 172.21.15.64:0/568591819 wait complete. 2024-04-01T00:14:25.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:25.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.732+0000 7f8c11ec0640 1 Processor -- start 2024-04-01T00:14:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.732+0000 7f8c11ec0640 1 -- start start 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c11ec0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8c0c1599b0 0x7f8c0c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c11ec0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c0c160210 0x7f8c0c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c11ec0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0b3a10 con 0x7f8c0c0b3500 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c11ec0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c06cc50 con 0x7f8c0c160210 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c11ec0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0b3cf0 con 0x7f8c0c1599b0 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c0c160210 0x7f8c0c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8c0c0b3500 0x7f8c0c15fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42394/0 (socket says 172.21.15.64:42394) 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0affd640 1 -- 172.21.15.64:0/3127483630 learned_addr learned my addr 172.21.15.64:0/3127483630 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0b7fe640 1 --2- 172.21.15.64:0/3127483630 >> v2:172.21.15.100:3300/0 conn(0x7f8c0c1599b0 0x7f8c0c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3127483630 >> v2:172.21.15.100:3300/0 conn(0x7f8c0c1599b0 msgr2=0x7f8c0c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0bfff640 1 --2- 172.21.15.64:0/3127483630 >> v2:172.21.15.100:3300/0 conn(0x7f8c0c1599b0 0x7f8c0c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3127483630 >> v1:172.21.15.64:6789/0 conn(0x7f8c0c0b3500 legacy=0x7f8c0c15fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3127483630 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c0b3fd0 con 0x7f8c0c160210 2024-04-01T00:14:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.733+0000 7f8c0a7fc640 1 -- 172.21.15.64:0/3127483630 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543913803 0 0) 0x7f8c0c0b3a10 con 0x7f8c0c0b3500 2024-04-01T00:14:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.734+0000 7f8c0bfff640 1 --2- 172.21.15.64:0/3127483630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c0ce7d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0d0c20 tx=0x7f8c0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb26d5d4f8de75f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.734+0000 7f8c0a7fc640 1 -- 172.21.15.64:0/3127483630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04039070 con 0x7f8c0c160210 2024-04-01T00:14:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.734+0000 7f8c0a7fc640 1 -- 172.21.15.64:0/3127483630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0402cc80 con 0x7f8c0c160210 2024-04-01T00:14:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.734+0000 7f8c0a7fc640 1 -- 172.21.15.64:0/3127483630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04034660 con 0x7f8c0c160210 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3127483630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 msgr2=0x7f8c0c0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3127483630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c0ce7d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0d0c20 tx=0x7f8c0402bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3127483630 shutdown_connections 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3127483630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c0ce7d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3127483630 >> v2:172.21.15.100:3300/0 conn(0x7f8c0c1599b0 0x7f8c0c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3127483630 >> 172.21.15.64:0/3127483630 conn(0x7f8c0c0b1730 msgr2=0x7f8c0c0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3127483630 shutdown_connections 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.735+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3127483630 wait complete. 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.736+0000 7f8c11ec0640 1 Processor -- start 2024-04-01T00:14:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.736+0000 7f8c11ec0640 1 -- start start 2024-04-01T00:14:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c0c0b3500 0x7f8c0c0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0c1599b0 0x7f8c0c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c159d90 con 0x7f8c0c160210 2024-04-01T00:14:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0ce280 con 0x7f8c0c1599b0 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c14c240 con 0x7f8c0c0b3500 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46934/0 (socket says 172.21.15.64:46934) 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3519781793 learned_addr learned my addr 172.21.15.64:0/3519781793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0affd640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0c1599b0 0x7f8c0c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0b7fe640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c0c0b3500 0x7f8c0c0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c0c0b3500 msgr2=0x7f8c0c0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c0c0b3500 0x7f8c0c0b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0c1599b0 msgr2=0x7f8c0c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0c1599b0 0x7f8c0c0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 -- 172.21.15.64:0/3519781793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c0d0710 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c0bfff640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009e80 tx=0x7f8c04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04039050 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c04045070 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1f7a30 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.737+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04009450 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.738+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1f7ff0 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.738+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c0681a0 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.739+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f8c0403e850 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.739+0000 7f8bf3fff640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 0x7f8bf806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.739+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c040ad5b0 con 0x7f8c0c160210 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.739+0000 7f8c0b7fe640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 0x7f8bf806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.740+0000 7f8c0b7fe640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 0x7f8bf806de50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009b40 tx=0x7f8bfc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.744+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8c0407bc90 con 0x7f8c0c160210 2024-04-01T00:14:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.923+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c0c0b0830 con 0x7f8c0c160210 2024-04-01T00:14:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.924+0000 7f8bf3fff640 1 -- 172.21.15.64:0/3519781793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8c0407fd30 con 0x7f8c0c160210 2024-04-01T00:14:25.924 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:25.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 msgr2=0x7f8bf806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 0x7f8bf806de50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009b40 tx=0x7f8bfc009190 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 msgr2=0x7f8c0c148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009e80 tx=0x7f8c04004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 shutdown_connections 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8bf806b990 0x7f8bf806de50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c0c160210 0x7f8c0c148380 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0c1599b0 0x7f8c0c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 --2- 172.21.15.64:0/3519781793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c0c0b3500 0x7f8c0c0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.925+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 >> 172.21.15.64:0/3519781793 conn(0x7f8c0c0b1730 msgr2=0x7f8c0c0b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.926+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 shutdown_connections 2024-04-01T00:14:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:25.926+0000 7f8c11ec0640 1 -- 172.21.15.64:0/3519781793 wait complete. 2024-04-01T00:14:25.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.042+0000 7fcf80365640 1 Processor -- start 2024-04-01T00:14:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.042+0000 7fcf80365640 1 -- start start 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf80365640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf780726c0 0x7fcf78169010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf80365640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf780b0570 con 0x7fcf7815e120 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf80365640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf780c6040 con 0x7fcf7806b7d0 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf80365640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fcf78072ad0 con 0x7fcf780726c0 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e902640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf780726c0 0x7fcf78169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e101640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcf7815e120 0x7fcf7806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42398/0 (socket says 172.21.15.64:42398) 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e101640 1 -- 172.21.15.64:0/3104006815 learned_addr learned my addr 172.21.15.64:0/3104006815 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e902640 1 -- 172.21.15.64:0/3104006815 >> v1:172.21.15.100:6789/0 conn(0x7fcf7806b7d0 legacy=0x7fcf78071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e902640 1 -- 172.21.15.64:0/3104006815 >> v1:172.21.15.64:6789/0 conn(0x7fcf7815e120 legacy=0x7fcf7806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7e902640 1 -- 172.21.15.64:0/3104006815 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf780c6320 con 0x7fcf780726c0 2024-04-01T00:14:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.043+0000 7fcf7d0ff640 1 -- 172.21.15.64:0/3104006815 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902332490 0 0) 0x7fcf780b0570 con 0x7fcf7815e120 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.044+0000 7fcf7e902640 1 --2- 172.21.15.64:0/3104006815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf78169010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009d90 tx=0x7fcf7002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f59fb3b7b947049 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.044+0000 7fcf7d0ff640 1 -- 172.21.15.64:0/3104006815 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70039070 con 0x7fcf780726c0 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.044+0000 7fcf7d0ff640 1 -- 172.21.15.64:0/3104006815 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7002cd00 con 0x7fcf780726c0 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.044+0000 7fcf7d0ff640 1 -- 172.21.15.64:0/3104006815 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70034660 con 0x7fcf780726c0 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 -- 172.21.15.64:0/3104006815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 msgr2=0x7fcf78169010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 --2- 172.21.15.64:0/3104006815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf78169010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009d90 tx=0x7fcf7002bf50 comp rx=0 tx=0).stop 2024-04-01T00:14:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 -- 172.21.15.64:0/3104006815 shutdown_connections 2024-04-01T00:14:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 --2- 172.21.15.64:0/3104006815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf78169010 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 -- 172.21.15.64:0/3104006815 >> 172.21.15.64:0/3104006815 conn(0x7fcf780d0ba0 msgr2=0x7fcf780d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 -- 172.21.15.64:0/3104006815 shutdown_connections 2024-04-01T00:14:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.045+0000 7fcf80365640 1 -- 172.21.15.64:0/3104006815 wait complete. 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.046+0000 7fcf80365640 1 Processor -- start 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.046+0000 7fcf80365640 1 -- start start 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf7806b7d0 0x7fcf781467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf7815e120 0x7fcf780b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780c68e0 con 0x7fcf780726c0 2024-04-01T00:14:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78072d10 con 0x7fcf7806b7d0 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf780cc2b0 con 0x7fcf7815e120 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7e101640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf7806b7d0 0x7fcf781467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46952/0 (socket says 172.21.15.64:46952) 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 -- 172.21.15.64:0/69671367 learned_addr learned my addr 172.21.15.64:0/69671367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7e902640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf7815e120 0x7fcf780b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 -- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf7815e120 msgr2=0x7fcf780b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf7815e120 0x7fcf780b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 -- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf7806b7d0 msgr2=0x7fcf781467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf7806b7d0 0x7fcf781467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 -- 172.21.15.64:0/69671367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781613a0 con 0x7fcf780726c0 2024-04-01T00:14:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf7d900640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c450 tx=0x7fcf6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680052d0 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68005470 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf78160820 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.047+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68012870 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.048+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78160de0 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.048+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf780681a0 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.049+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fcf68012a10 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.049+0000 7fcf6effd640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 0x7fcf5c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.049+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf6808a2f0 con 0x7fcf780726c0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.049+0000 7fcf7e101640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 0x7fcf5c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.050+0000 7fcf7e101640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 0x7fcf5c06de50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf74000f60 tx=0x7fcf74007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.054+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fcf680588b0 con 0x7fcf780726c0 2024-04-01T00:14:26.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.235+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf781615f0 con 0x7fcf780726c0 2024-04-01T00:14:26.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.235+0000 7fcf6effd640 1 -- 172.21.15.64:0/69671367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fcf6805c950 con 0x7fcf780726c0 2024-04-01T00:14:26.236 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 msgr2=0x7fcf5c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 0x7fcf5c06de50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf74000f60 tx=0x7fcf74007e40 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 msgr2=0x7fcf780b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c450 tx=0x7fcf6800c910 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 shutdown_connections 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcf5c06b990 0x7fcf5c06de50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf7815e120 0x7fcf780b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf780726c0 0x7fcf780b1870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 --2- 172.21.15.64:0/69671367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf7806b7d0 0x7fcf781467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 >> 172.21.15.64:0/69671367 conn(0x7fcf780d0ba0 msgr2=0x7fcf780c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 shutdown_connections 2024-04-01T00:14:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.237+0000 7fcf80365640 1 -- 172.21.15.64:0/69671367 wait complete. 2024-04-01T00:14:26.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 Processor -- start 2024-04-01T00:14:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 -- start start 2024-04-01T00:14:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 0x7fe8380b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8380b3f10 0x7fe838161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8380d2800 con 0x7fe8380b5710 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe8380d2aa0 con 0x7fe8380b3f10 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.353+0000 7fe83df25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe8380d2d80 con 0x7fe8380b3b30 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8380b3f10 0x7fe838161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe836ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe8380b5710 0x7fe8380b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42400/0 (socket says 172.21.15.64:42400) 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8380b3f10 0x7fe838161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46958/0 (socket says 172.21.15.64:46958) 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe8377fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 0x7fe8380b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 -- 172.21.15.64:0/2264844140 learned_addr learned my addr 172.21.15.64:0/2264844140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 -- 172.21.15.64:0/2264844140 >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 msgr2=0x7fe8380b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 --2- 172.21.15.64:0/2264844140 >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 0x7fe8380b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 -- 172.21.15.64:0/2264844140 >> v1:172.21.15.64:6789/0 conn(0x7fe8380b5710 legacy=0x7fe8380b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe8367fc640 1 -- 172.21.15.64:0/2264844140 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037476563 0 0) 0x7fe8380d2800 con 0x7fe8380b5710 2024-04-01T00:14:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe837fff640 1 -- 172.21.15.64:0/2264844140 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8380b4320 con 0x7fe8380b3f10 2024-04-01T00:14:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.354+0000 7fe8377fe640 1 --2- 172.21.15.64:0/2264844140 >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 0x7fe8380b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe837fff640 1 --2- 172.21.15.64:0/2264844140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3f10 0x7fe838161460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe83808e470 tx=0x7fe83002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b264a4039107f7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe8367fc640 1 -- 172.21.15.64:0/2264844140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83002cb20 con 0x7fe8380b3f10 2024-04-01T00:14:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe8367fc640 1 -- 172.21.15.64:0/2264844140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83002ccc0 con 0x7fe8380b3f10 2024-04-01T00:14:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe8367fc640 1 -- 172.21.15.64:0/2264844140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830034610 con 0x7fe8380b3f10 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 -- 172.21.15.64:0/2264844140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3f10 msgr2=0x7fe838161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 --2- 172.21.15.64:0/2264844140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3f10 0x7fe838161460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe83808e470 tx=0x7fe83002bf80 comp rx=0 tx=0).stop 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 -- 172.21.15.64:0/2264844140 shutdown_connections 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 --2- 172.21.15.64:0/2264844140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3f10 0x7fe838161460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 --2- 172.21.15.64:0/2264844140 >> v2:172.21.15.100:3300/0 conn(0x7fe8380b3b30 0x7fe8380b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 -- 172.21.15.64:0/2264844140 >> 172.21.15.64:0/2264844140 conn(0x7fe83809d3f0 msgr2=0x7fe8380d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 -- 172.21.15.64:0/2264844140 shutdown_connections 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.355+0000 7fe83df25640 1 -- 172.21.15.64:0/2264844140 wait complete. 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 Processor -- start 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 -- start start 2024-04-01T00:14:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8380b3f10 0x7fe838156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8380b5710 0x7fe838160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe838161b20 con 0x7fe8380b3b30 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381619a0 con 0x7fe8380b5710 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.357+0000 7fe83df25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe838161ca0 con 0x7fe8380b3f10 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46966/0 (socket says 172.21.15.64:46966) 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 -- 172.21.15.64:0/3093099730 learned_addr learned my addr 172.21.15.64:0/3093099730 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe836ffd640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8380b3f10 0x7fe838156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe837fff640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8380b5710 0x7fe838160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 -- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8380b3f10 msgr2=0x7fe838156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8380b3f10 0x7fe838156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 -- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8380b5710 msgr2=0x7fe838160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8380b5710 0x7fe838160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 -- 172.21.15.64:0/3093099730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe838156b80 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe8377fe640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe82800e990 tx=0x7fe82800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828017070 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82800c9e0 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8381f9d70 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82800cdb0 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.358+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8381fa330 con 0x7fe8380b3b30 2024-04-01T00:14:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.359+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8380681a0 con 0x7fe8380b3b30 2024-04-01T00:14:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.360+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe82801a070 con 0x7fe8380b3b30 2024-04-01T00:14:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.361+0000 7fe81ffff640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 0x7fe82406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.361+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe82808aa80 con 0x7fe8380b3b30 2024-04-01T00:14:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.361+0000 7fe836ffd640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 0x7fe82406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.361+0000 7fe836ffd640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 0x7fe82406de50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8380a1fa0 tx=0x7fe82c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.365+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe838156b80 con 0x7fe8380b3b30 2024-04-01T00:14:26.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.546+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8380cae10 con 0x7fe8380b3b30 2024-04-01T00:14:26.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.546+0000 7fe81ffff640 1 -- 172.21.15.64:0/3093099730 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe828058fc0 con 0x7fe8380b3b30 2024-04-01T00:14:26.547 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:26.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 msgr2=0x7fe82406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 0x7fe82406de50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8380a1fa0 tx=0x7fe82c007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:26.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 msgr2=0x7fe8381fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe82800e990 tx=0x7fe82800ee50 comp rx=0 tx=0).stop 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 shutdown_connections 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe82406b990 0x7fe82406de50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8380b5710 0x7fe838160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8380b3f10 0x7fe838156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 --2- 172.21.15.64:0/3093099730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8380b3b30 0x7fe8381fae10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 >> 172.21.15.64:0/3093099730 conn(0x7fe83809d3f0 msgr2=0x7fe83806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 shutdown_connections 2024-04-01T00:14:26.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.548+0000 7fe83df25640 1 -- 172.21.15.64:0/3093099730 wait complete. 2024-04-01T00:14:26.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 Processor -- start 2024-04-01T00:14:26.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 -- start start 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e8415e230 0x7f8e84161d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 0x7f8e841622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e841627e0 0x7f8e840cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f8e840d0cc0 con 0x7f8e841627e0 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8e840d01a0 con 0x7f8e840cc5a0 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e89302640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8415e610 con 0x7f8e8415e230 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.662+0000 7f8e837fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e841627e0 0x7f8e840cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e841627e0 0x7f8e840cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46974/0 (socket says 172.21.15.64:46974) 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 -- 172.21.15.64:0/532699511 learned_addr learned my addr 172.21.15.64:0/532699511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e827fc640 1 --2- 172.21.15.64:0/532699511 >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 0x7f8e841622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e82ffd640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e8415e230 0x7f8e84161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 -- 172.21.15.64:0/532699511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e8415e230 msgr2=0x7f8e84161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e8415e230 0x7f8e84161d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 -- 172.21.15.64:0/532699511 >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 msgr2=0x7f8e841622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 --2- 172.21.15.64:0/532699511 >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 0x7f8e841622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e837fe640 1 -- 172.21.15.64:0/532699511 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840d0420 con 0x7f8e841627e0 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.663+0000 7f8e827fc640 1 --2- 172.21.15.64:0/532699511 >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 0x7f8e841622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e837fe640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e840cca80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8e840cdea0 tx=0x7f8e7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a84308f0f4694d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e81ffb640 1 -- 172.21.15.64:0/532699511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7402ca70 con 0x7f8e841627e0 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e81ffb640 1 -- 172.21.15.64:0/532699511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7402cc10 con 0x7f8e841627e0 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e81ffb640 1 -- 172.21.15.64:0/532699511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740343f0 con 0x7f8e841627e0 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 -- 172.21.15.64:0/532699511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 msgr2=0x7f8e840cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e840cca80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8e840cdea0 tx=0x7f8e7402bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 -- 172.21.15.64:0/532699511 shutdown_connections 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e840cca80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 --2- 172.21.15.64:0/532699511 >> v2:172.21.15.100:3300/0 conn(0x7f8e840cc5a0 0x7f8e841622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 --2- 172.21.15.64:0/532699511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e8415e230 0x7f8e84161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 -- 172.21.15.64:0/532699511 >> 172.21.15.64:0/532699511 conn(0x7f8e840719a0 msgr2=0x7f8e840b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 -- 172.21.15.64:0/532699511 shutdown_connections 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.664+0000 7f8e89302640 1 -- 172.21.15.64:0/532699511 wait complete. 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 Processor -- start 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 -- start start 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e840cc5a0 0x7f8e84099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8415e230 0x7f8e8406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e840cdf90 con 0x7f8e841627e0 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e840d09e0 con 0x7f8e8415e230 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e89302640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e840ce110 con 0x7f8e840cc5a0 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46978/0 (socket says 172.21.15.64:46978) 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.666+0000 7f8e837fe640 1 -- 172.21.15.64:0/758178537 learned_addr learned my addr 172.21.15.64:0/758178537 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e827fc640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8415e230 0x7f8e8406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e82ffd640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e840cc5a0 0x7f8e84099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 -- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e840cc5a0 msgr2=0x7f8e84099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e840cc5a0 0x7f8e84099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 -- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8415e230 msgr2=0x7f8e8406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8415e230 0x7f8e8406d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 -- 172.21.15.64:0/758178537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8414cc80 con 0x7f8e841627e0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e837fe640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e74036040 tx=0x7f8e74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74043020 con 0x7f8e841627e0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7403e450 con 0x7f8e841627e0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e840ca1a0 con 0x7f8e841627e0 2024-04-01T00:14:26.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74009510 con 0x7f8e841627e0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.667+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8414cf40 con 0x7f8e841627e0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.668+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e840681a0 con 0x7f8e841627e0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.668+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f8e74046020 con 0x7f8e841627e0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.669+0000 7f8e7b7fe640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 0x7f8e6406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.669+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e740accd0 con 0x7f8e841627e0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.669+0000 7f8e82ffd640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 0x7f8e6406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.669+0000 7f8e82ffd640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 0x7f8e6406de50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009aa0 tx=0x7f8e7c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:26.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.673+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8e7404d030 con 0x7f8e841627e0 2024-04-01T00:14:26.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.867+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e840d0ef0 con 0x7f8e841627e0 2024-04-01T00:14:26.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.867+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/758178537 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f8e7407b320 con 0x7f8e841627e0 2024-04-01T00:14:26.871 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:26.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 msgr2=0x7f8e6406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 0x7f8e6406de50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009aa0 tx=0x7f8e7c009190 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 msgr2=0x7f8e8406da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e74036040 tx=0x7f8e74004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 shutdown_connections 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8e6406b990 0x7f8e6406de50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e841627e0 0x7f8e8406da40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8415e230 0x7f8e8406d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 --2- 172.21.15.64:0/758178537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e840cc5a0 0x7f8e84099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 >> 172.21.15.64:0/758178537 conn(0x7f8e840719a0 msgr2=0x7f8e8406e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.869+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 shutdown_connections 2024-04-01T00:14:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:26.870+0000 7f8e89302640 1 -- 172.21.15.64:0/758178537 wait complete. 2024-04-01T00:14:26.882 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:14:26.882 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:26.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.039+0000 7fb2dffc9640 1 Processor -- start 2024-04-01T00:14:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.039+0000 7fb2dffc9640 1 -- start start 2024-04-01T00:14:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dffc9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 0x7fb2d80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dffc9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2d80cfae0 0x7fb2d81620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dffc9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d80d2ba0 con 0x7fb2d81619e0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dffc9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d80d2e80 con 0x7fb2d80cfae0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dffc9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d80ceeb0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2ddd65640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb2d81619e0 0x7fb2d80cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51848/0 (socket says 172.21.15.64:51848) 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2ddd65640 1 -- 172.21.15.64:0/594743404 learned_addr learned my addr 172.21.15.64:0/594743404 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2dd564640 1 --2- 172.21.15.64:0/594743404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 0x7fb2d80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.040+0000 7fb2de566640 1 --2- 172.21.15.64:0/594743404 >> v2:172.21.15.100:3300/0 conn(0x7fb2d80cfae0 0x7fb2d81620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dcd63640 1 -- 172.21.15.64:0/594743404 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378540225 0 0) 0x7fb2d80d2ba0 con 0x7fb2d81619e0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dcd63640 1 -- 172.21.15.64:0/594743404 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2c4003520 con 0x7fb2d81619e0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dd564640 1 -- 172.21.15.64:0/594743404 >> v2:172.21.15.100:3300/0 conn(0x7fb2d80cfae0 msgr2=0x7fb2d81620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dd564640 1 --2- 172.21.15.64:0/594743404 >> v2:172.21.15.100:3300/0 conn(0x7fb2d80cfae0 0x7fb2d81620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dd564640 1 -- 172.21.15.64:0/594743404 >> v1:172.21.15.100:6789/0 conn(0x7fb2d81619e0 legacy=0x7fb2d80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dd564640 1 -- 172.21.15.64:0/594743404 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d8161dc0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.041+0000 7fb2dd564640 1 --2- 172.21.15.64:0/594743404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 0x7fb2d80b3820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80097a0 tx=0x7fb2c802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f52968e90eb7697 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dcd63640 1 -- 172.21.15.64:0/594743404 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80315a0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dcd63640 1 -- 172.21.15.64:0/594743404 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c8037070 con 0x7fb2d80d4a60 2024-04-01T00:14:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dcd63640 1 -- 172.21.15.64:0/594743404 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8031960 con 0x7fb2d80d4a60 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 -- 172.21.15.64:0/594743404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 msgr2=0x7fb2d80b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/594743404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 0x7fb2d80b3820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80097a0 tx=0x7fb2c802b840 comp rx=0 tx=0).stop 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 -- 172.21.15.64:0/594743404 shutdown_connections 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/594743404 >> v2:172.21.15.100:3300/0 conn(0x7fb2d80cfae0 0x7fb2d81620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/594743404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80d4a60 0x7fb2d80b3820 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 -- 172.21.15.64:0/594743404 >> 172.21.15.64:0/594743404 conn(0x7fb2d809d290 msgr2=0x7fb2d80d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 -- 172.21.15.64:0/594743404 shutdown_connections 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.042+0000 7fb2dffc9640 1 -- 172.21.15.64:0/594743404 wait complete. 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.043+0000 7fb2dffc9640 1 Processor -- start 2024-04-01T00:14:27.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.043+0000 7fb2dffc9640 1 -- start start 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80cfae0 0x7fb2d81f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d81619e0 0x7fb2d814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80ceb10 con 0x7fb2d80d4a60 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80b31b0 con 0x7fb2d81619e0 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dffc9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d80cec90 con 0x7fb2d80cfae0 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dd564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2ddd65640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80cfae0 0x7fb2d81f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dd564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46980/0 (socket says 172.21.15.64:46980) 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2dd564640 1 -- 172.21.15.64:0/3965838789 learned_addr learned my addr 172.21.15.64:0/3965838789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.044+0000 7fb2de566640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d81619e0 0x7fb2d814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 -- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80cfae0 msgr2=0x7fb2d81f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80cfae0 0x7fb2d81f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 -- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d81619e0 msgr2=0x7fb2d814dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d81619e0 0x7fb2d814dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 -- 172.21.15.64:0/3965838789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d80cb080 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dd564640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8034040 tx=0x7fb2c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8032630 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c80327d0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d814e910 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8032bc0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.045+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d814eed0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.046+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d80681a0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.046+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb2c803b3f0 con 0x7fb2d80d4a60 2024-04-01T00:14:27.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.047+0000 7fb2ce7fc640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 0x7fb2bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.047+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c802f030 con 0x7fb2d80d4a60 2024-04-01T00:14:27.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.047+0000 7fb2ddd65640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 0x7fb2bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.047+0000 7fb2ddd65640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 0x7fb2bc06de50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005880 tx=0x7fb2d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.051+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb2c8040070 con 0x7fb2d80d4a60 2024-04-01T00:14:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.236+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2d809d730 con 0x7fb2d80d4a60 2024-04-01T00:14:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.237+0000 7fb2ce7fc640 1 -- 172.21.15.64:0/3965838789 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16830 (secure 0 0 0) 0x7fb2c807aa00 con 0x7fb2d80d4a60 2024-04-01T00:14:27.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:27.110609+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 msgr2=0x7fb2bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 0x7fb2bc06de50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005880 tx=0x7fb2d4009040 comp rx=0 tx=0).stop 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 msgr2=0x7fb2d81f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8034040 tx=0x7fb2c80046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 shutdown_connections 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2bc06b990 0x7fb2bc06de50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d81619e0 0x7fb2d814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d80d4a60 0x7fb2d81f75d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 --2- 172.21.15.64:0/3965838789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2d80cfae0 0x7fb2d81f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 >> 172.21.15.64:0/3965838789 conn(0x7fb2d809d290 msgr2=0x7fb2d80c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 shutdown_connections 2024-04-01T00:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.240+0000 7fb2dffc9640 1 -- 172.21.15.64:0/3965838789 wait complete. 2024-04-01T00:14:27.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:27.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:27.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.356+0000 7f71f8a55640 1 Processor -- start 2024-04-01T00:14:27.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.356+0000 7f71f8a55640 1 -- start start 2024-04-01T00:14:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f8a55640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71f415e260 0x7f71f40d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f8a55640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f40cf400 con 0x7f71f415e640 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f8a55640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f40d0b80 con 0x7f71f40b2f00 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f8a55640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f71f40d11d0 con 0x7f71f415e260 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f1d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f71f415e640 0x7f71f40ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42410/0 (socket says 172.21.15.64:42410) 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f1d9b640 1 -- 172.21.15.64:0/1795337854 learned_addr learned my addr 172.21.15.64:0/1795337854 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f259c640 1 --2- 172.21.15.64:0/1795337854 >> v2:172.21.15.100:3300/0 conn(0x7f71f415e260 0x7f71f40d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f159a640 1 -- 172.21.15.64:0/1795337854 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258404820 0 0) 0x7f71f40cf400 con 0x7f71f415e640 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f159a640 1 -- 172.21.15.64:0/1795337854 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e0003520 con 0x7f71f415e640 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.357+0000 7f71f259c640 1 -- 172.21.15.64:0/1795337854 >> v1:172.21.15.100:6789/0 conn(0x7f71f40b2f00 legacy=0x7f71f40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f259c640 1 -- 172.21.15.64:0/1795337854 >> v1:172.21.15.64:6789/0 conn(0x7f71f415e640 legacy=0x7f71f40ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f259c640 1 -- 172.21.15.64:0/1795337854 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f40d0e60 con 0x7f71f415e260 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f259c640 1 --2- 172.21.15.64:0/1795337854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f415e260 0x7f71f40d2890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998e5cd23ca8f567 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f159a640 1 -- 172.21.15.64:0/1795337854 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02f810 con 0x7f71f415e260 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f159a640 1 -- 172.21.15.64:0/1795337854 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec02f9b0 con 0x7f71f415e260 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.358+0000 7f71f159a640 1 -- 172.21.15.64:0/1795337854 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02fd80 con 0x7f71f415e260 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 -- 172.21.15.64:0/1795337854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f415e260 msgr2=0x7f71f40d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 --2- 172.21.15.64:0/1795337854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f415e260 0x7f71f40d2890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02b9f0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 -- 172.21.15.64:0/1795337854 shutdown_connections 2024-04-01T00:14:27.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 --2- 172.21.15.64:0/1795337854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f415e260 0x7f71f40d2890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 -- 172.21.15.64:0/1795337854 >> 172.21.15.64:0/1795337854 conn(0x7f71f40d1d70 msgr2=0x7f71f40d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 -- 172.21.15.64:0/1795337854 shutdown_connections 2024-04-01T00:14:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.359+0000 7f71f8a55640 1 -- 172.21.15.64:0/1795337854 wait complete. 2024-04-01T00:14:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.360+0000 7f71f8a55640 1 Processor -- start 2024-04-01T00:14:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.360+0000 7f71f8a55640 1 -- start start 2024-04-01T00:14:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 0x7f71f4147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71f415e640 0x7f71f4147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f71f40cf110 con 0x7f71f415e260 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71f40cef90 con 0x7f71f40b2f00 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f8a55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f71f40ccc80 con 0x7f71f415e640 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46990/0 (socket says 172.21.15.64:46990) 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f1d9b640 1 -- 172.21.15.64:0/2286117256 learned_addr learned my addr 172.21.15.64:0/2286117256 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f259c640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 0x7f71f4147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.361+0000 7f71f2d9d640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71f415e640 0x7f71f4147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 -- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71f415e640 msgr2=0x7f71f4147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71f415e640 0x7f71f4147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 -- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 msgr2=0x7f71f4147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 0x7f71f4147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 -- 172.21.15.64:0/2286117256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f414a930 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f259c640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 0x7f71f4147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f1d9b640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00c410 tx=0x7f71dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc017070 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f414aec0 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc005ce0 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0072f0 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.362+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f40ca9d0 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.363+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f40681a0 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.363+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f71dc01a070 con 0x7f71f415e260 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.364+0000 7f71eaffd640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 0x7f71d406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.364+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f71dc089c10 con 0x7f71f415e260 2024-04-01T00:14:27.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.364+0000 7f71f259c640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 0x7f71d406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.364+0000 7f71f259c640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 0x7f71d406de80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.368+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f71dc058220 con 0x7f71f415e260 2024-04-01T00:14:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.548+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71f40d0500 con 0x7f71f415e260 2024-04-01T00:14:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.548+0000 7f71eaffd640 1 -- 172.21.15.64:0/2286117256 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f71dc05c2c0 con 0x7f71f415e260 2024-04-01T00:14:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 msgr2=0x7f71d406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 0x7f71d406de80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec008030 comp rx=0 tx=0).stop 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 msgr2=0x7f71f41e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00c410 tx=0x7f71dc00c8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 shutdown_connections 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f71d406b9c0 0x7f71d406de80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71f415e640 0x7f71f4147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71f415e260 0x7f71f41e84c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 --2- 172.21.15.64:0/2286117256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f40b2f00 0x7f71f4147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 >> 172.21.15.64:0/2286117256 conn(0x7f71f40d1d70 msgr2=0x7f71f406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 shutdown_connections 2024-04-01T00:14:27.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.550+0000 7f71f8a55640 1 -- 172.21.15.64:0/2286117256 wait complete. 2024-04-01T00:14:27.562 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:27.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.666+0000 7f1fd305c640 1 Processor -- start 2024-04-01T00:14:27.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.666+0000 7f1fd305c640 1 -- start start 2024-04-01T00:14:27.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd305c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 0x7f1fcc0d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd305c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fcc0d0900 0x7f1fcc0d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd305c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fcc161e10 con 0x7f1fcc0b3740 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd305c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1fcc0b3ba0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd305c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0cef20 con 0x7f1fcc161a30 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fcbfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1fcc0b3740 0x7f1fcc0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51858/0 (socket says 172.21.15.64:51858) 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fcbfff640 1 -- 172.21.15.64:0/2193192420 learned_addr learned my addr 172.21.15.64:0/2193192420 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/2193192420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 0x7f1fcc0d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.667+0000 7f1fd15f9640 1 --2- 172.21.15.64:0/2193192420 >> v2:172.21.15.100:3300/0 conn(0x7f1fcc0d0900 0x7f1fcc0d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fd0df8640 1 -- 172.21.15.64:0/2193192420 >> v2:172.21.15.100:3300/0 conn(0x7f1fcc0d0900 msgr2=0x7f1fcc0d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fcb7fe640 1 -- 172.21.15.64:0/2193192420 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195319102 0 0) 0x7f1fb4001110 con 0x7f1fcc0b3740 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/2193192420 >> v2:172.21.15.100:3300/0 conn(0x7f1fcc0d0900 0x7f1fcc0d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fd0df8640 1 -- 172.21.15.64:0/2193192420 >> v1:172.21.15.100:6789/0 conn(0x7f1fcc0b3740 legacy=0x7f1fcc0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fd0df8640 1 -- 172.21.15.64:0/2193192420 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc0d2880 con 0x7f1fcc161a30 2024-04-01T00:14:27.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.668+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/2193192420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 0x7f1fcc0d4a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009d90 tx=0x7f1fc402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3be8a3235cf33c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fcb7fe640 1 -- 172.21.15.64:0/2193192420 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4039070 con 0x7f1fcc161a30 2024-04-01T00:14:27.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fcb7fe640 1 -- 172.21.15.64:0/2193192420 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc402cc50 con 0x7f1fcc161a30 2024-04-01T00:14:27.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fcb7fe640 1 -- 172.21.15.64:0/2193192420 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4034440 con 0x7f1fcc161a30 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 -- 172.21.15.64:0/2193192420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 msgr2=0x7f1fcc0d4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 --2- 172.21.15.64:0/2193192420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 0x7f1fcc0d4a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009d90 tx=0x7f1fc402bea0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 -- 172.21.15.64:0/2193192420 shutdown_connections 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 --2- 172.21.15.64:0/2193192420 >> v2:172.21.15.100:3300/0 conn(0x7f1fcc0d0900 0x7f1fcc0d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 --2- 172.21.15.64:0/2193192420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc161a30 0x7f1fcc0d4a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 -- 172.21.15.64:0/2193192420 >> 172.21.15.64:0/2193192420 conn(0x7f1fcc09d250 msgr2=0x7f1fcc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 -- 172.21.15.64:0/2193192420 shutdown_connections 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.669+0000 7f1fd305c640 1 -- 172.21.15.64:0/2193192420 wait complete. 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.670+0000 7f1fd305c640 1 Processor -- start 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.670+0000 7f1fd305c640 1 -- start start 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 0x7f1fcc15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc161a30 0x7f1fcc0b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0ceb40 con 0x7f1fcc0d0900 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0d2e00 con 0x7f1fcc161a30 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd305c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0cf3b0 con 0x7f1fcc0b3740 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47006/0 (socket says 172.21.15.64:47006) 2024-04-01T00:14:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fcbfff640 1 -- 172.21.15.64:0/1751755685 learned_addr learned my addr 172.21.15.64:0/1751755685 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.671+0000 7f1fd0df8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 0x7f1fcc15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fd15f9640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc161a30 0x7f1fcc0b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 -- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 msgr2=0x7f1fcc15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 0x7f1fcc15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 -- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc161a30 msgr2=0x7f1fcc0b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc161a30 0x7f1fcc0b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 -- 172.21.15.64:0/1751755685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc15f5d0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 0x7f1fcc15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fcbfff640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c780 tx=0x7f1fb400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4018070 con 0x7f1fcc0d0900 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb40072d0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc15f8b0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40047e0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.672+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc15fe70 con 0x7f1fcc0d0900 2024-04-01T00:14:27.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.673+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc0681a0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.674+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1fb4004980 con 0x7f1fcc0d0900 2024-04-01T00:14:27.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.674+0000 7f1fc97fa640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 0x7f1fac06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.674+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f1fb4089b30 con 0x7f1fcc0d0900 2024-04-01T00:14:27.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.674+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 0x7f1fac06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.675+0000 7f1fd0df8640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 0x7f1fac06de50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4036000 tx=0x7f1fc4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.679+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1fcc161e10 con 0x7f1fcc0d0900 2024-04-01T00:14:27.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.857+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1fcc09d730 con 0x7f1fcc0d0900 2024-04-01T00:14:27.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.858+0000 7f1fc97fa640 1 -- 172.21.15.64:0/1751755685 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f1fb4057ff0 con 0x7f1fcc0d0900 2024-04-01T00:14:27.859 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:27.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.859+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 msgr2=0x7f1fac06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.859+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 0x7f1fac06de50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4036000 tx=0x7f1fc4030000 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.859+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 msgr2=0x7f1fcc0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.859+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c780 tx=0x7f1fb400cc40 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 shutdown_connections 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fac06b990 0x7f1fac06de50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc161a30 0x7f1fcc0b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc0d0900 0x7f1fcc0b8cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 --2- 172.21.15.64:0/1751755685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0b3740 0x7f1fcc15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 >> 172.21.15.64:0/1751755685 conn(0x7f1fcc09d250 msgr2=0x7f1fcc0cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 shutdown_connections 2024-04-01T00:14:27.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.860+0000 7f1fd305c640 1 -- 172.21.15.64:0/1751755685 wait complete. 2024-04-01T00:14:27.871 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.975+0000 7f2afe4c0640 1 Processor -- start 2024-04-01T00:14:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.975+0000 7f2afe4c0640 1 -- start start 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afe4c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afe4c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2af80d2780 0x7f2af80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afe4c0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80cc350 con 0x7f2af80b2f20 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afe4c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2af80d0aa0 con 0x7f2af80d2780 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afe4c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80d0cf0 con 0x7f2af815e250 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afca5d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2af80b2f20 0x7f2af806c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42424/0 (socket says 172.21.15.64:42424) 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2afca5d640 1 -- 172.21.15.64:0/3609501184 learned_addr learned my addr 172.21.15.64:0/3609501184 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af7fff640 1 --2- 172.21.15.64:0/3609501184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af77fe640 1 --2- 172.21.15.64:0/3609501184 >> v2:172.21.15.100:3300/0 conn(0x7f2af80d2780 0x7f2af80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af6ffd640 1 -- 172.21.15.64:0/3609501184 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1543585375 0 0) 0x7f2af80cc350 con 0x7f2af80b2f20 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af6ffd640 1 -- 172.21.15.64:0/3609501184 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae0003520 con 0x7f2af80b2f20 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af7fff640 1 -- 172.21.15.64:0/3609501184 >> v2:172.21.15.100:3300/0 conn(0x7f2af80d2780 msgr2=0x7f2af80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af7fff640 1 --2- 172.21.15.64:0/3609501184 >> v2:172.21.15.100:3300/0 conn(0x7f2af80d2780 0x7f2af80cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af7fff640 1 -- 172.21.15.64:0/3609501184 >> v1:172.21.15.64:6789/0 conn(0x7f2af80b2f20 legacy=0x7f2af806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.976+0000 7f2af7fff640 1 -- 172.21.15.64:0/3609501184 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af80d0fd0 con 0x7f2af815e250 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.977+0000 7f2af7fff640 1 --2- 172.21.15.64:0/3609501184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80ce4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097a0 tx=0x7f2ae802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd41e10d8b38596f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.977+0000 7f2af6ffd640 1 -- 172.21.15.64:0/3609501184 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80315a0 con 0x7f2af815e250 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.977+0000 7f2af6ffd640 1 -- 172.21.15.64:0/3609501184 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae8037070 con 0x7f2af815e250 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.977+0000 7f2af6ffd640 1 -- 172.21.15.64:0/3609501184 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8031960 con 0x7f2af815e250 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 -- 172.21.15.64:0/3609501184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 msgr2=0x7f2af80ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/3609501184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80ce4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097a0 tx=0x7f2ae802b840 comp rx=0 tx=0).stop 2024-04-01T00:14:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 -- 172.21.15.64:0/3609501184 shutdown_connections 2024-04-01T00:14:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/3609501184 >> v2:172.21.15.100:3300/0 conn(0x7f2af80d2780 0x7f2af80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/3609501184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80ce4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 -- 172.21.15.64:0/3609501184 >> 172.21.15.64:0/3609501184 conn(0x7f2af80d1c60 msgr2=0x7f2af80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 -- 172.21.15.64:0/3609501184 shutdown_connections 2024-04-01T00:14:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.978+0000 7f2afe4c0640 1 -- 172.21.15.64:0/3609501184 wait complete. 2024-04-01T00:14:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.979+0000 7f2afe4c0640 1 Processor -- start 2024-04-01T00:14:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.979+0000 7f2afe4c0640 1 -- start start 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af80b2f20 0x7f2af8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2af802ce60 con 0x7f2af80d2780 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2af802cce0 con 0x7f2af80b2f20 2024-04-01T00:14:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afe4c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80cd810 con 0x7f2af815e250 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47022/0 (socket says 172.21.15.64:47022) 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af80b2f20 0x7f2af8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 -- 172.21.15.64:0/1119267207 learned_addr learned my addr 172.21.15.64:0/1119267207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2afca5d640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 -- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 msgr2=0x7f2af80af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 -- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af80b2f20 msgr2=0x7f2af8152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af80b2f20 0x7f2af8152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 -- 172.21.15.64:0/1119267207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81f7040 con 0x7f2af80d2780 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af77fe640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9b0 tx=0x7f2aec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec017070 con 0x7f2af80d2780 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.980+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec00c9a0 con 0x7f2af80d2780 2024-04-01T00:14:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.981+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af81f7320 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.981+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00cd90 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.981+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81f78e0 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.982+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af80681a0 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.982+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2aec01a070 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.983+0000 7f2af4ff9640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 0x7f2ae406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.983+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f2aec08ab70 con 0x7f2af80d2780 2024-04-01T00:14:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.983+0000 7f2af7fff640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 0x7f2ae406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:27.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.983+0000 7f2af7fff640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 0x7f2ae406de50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8036000 tx=0x7f2ae803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:27.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:27.987+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2aec059140 con 0x7f2af80d2780 2024-04-01T00:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.170+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2af80ce680 con 0x7f2af80d2780 2024-04-01T00:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.171+0000 7f2af4ff9640 1 -- 172.21.15.64:0/1119267207 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2aec05d1e0 con 0x7f2af80d2780 2024-04-01T00:14:28.172 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 msgr2=0x7f2ae406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 0x7f2ae406de50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8036000 tx=0x7f2ae803d040 comp rx=0 tx=0).stop 2024-04-01T00:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 msgr2=0x7f2af80af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9b0 tx=0x7f2aec00ee70 comp rx=0 tx=0).stop 2024-04-01T00:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 shutdown_connections 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ae406b990 0x7f2ae406de50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af815e250 0x7f2af80af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af80d2780 0x7f2af80af140 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 --2- 172.21.15.64:0/1119267207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af80b2f20 0x7f2af8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 >> 172.21.15.64:0/1119267207 conn(0x7f2af80d1c60 msgr2=0x7f2af815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 shutdown_connections 2024-04-01T00:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.173+0000 7f2afe4c0640 1 -- 172.21.15.64:0/1119267207 wait complete. 2024-04-01T00:14:28.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:28.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.288+0000 7f49f7ed4640 1 Processor -- start 2024-04-01T00:14:28.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 -- start start 2024-04-01T00:14:28.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f49f0161a30 0x7f49f00d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49f00d0900 0x7f49f00d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f0161e10 con 0x7f49f00b3740 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f49f00b3ba0 con 0x7f49f00d0900 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f7ed4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f49f00cef20 con 0x7f49f0161a30 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f6471640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49f00d0900 0x7f49f00d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f6471640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49f00d0900 0x7f49f00d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47036/0 (socket says 172.21.15.64:47036) 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f546f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f49f00b3740 0x7f49f00b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42430/0 (socket says 172.21.15.64:42430) 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.289+0000 7f49f6471640 1 -- 172.21.15.64:0/3894314512 learned_addr learned my addr 172.21.15.64:0/3894314512 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:28.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f5c70640 1 --2- 172.21.15.64:0/3894314512 >> v2:172.21.15.100:3300/0 conn(0x7f49f0161a30 0x7f49f00d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f6471640 1 -- 172.21.15.64:0/3894314512 >> v2:172.21.15.100:3300/0 conn(0x7f49f0161a30 msgr2=0x7f49f00d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f6471640 1 --2- 172.21.15.64:0/3894314512 >> v2:172.21.15.100:3300/0 conn(0x7f49f0161a30 0x7f49f00d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f6471640 1 -- 172.21.15.64:0/3894314512 >> v1:172.21.15.64:6789/0 conn(0x7f49f00b3740 legacy=0x7f49f00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f6471640 1 -- 172.21.15.64:0/3894314512 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00d2880 con 0x7f49f00d0900 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f4c6e640 1 -- 172.21.15.64:0/3894314512 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391971061 0 0) 0x7f49e0001110 con 0x7f49f00b3740 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.290+0000 7f49f6471640 1 --2- 172.21.15.64:0/3894314512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00d0900 0x7f49f00d0db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009e50 tx=0x7f49ec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb8fa6cc9a0fb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f4c6e640 1 -- 172.21.15.64:0/3894314512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec02cb20 con 0x7f49f00d0900 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f4c6e640 1 -- 172.21.15.64:0/3894314512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec02ccc0 con 0x7f49f00d0900 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f4c6e640 1 -- 172.21.15.64:0/3894314512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec034610 con 0x7f49f00d0900 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 -- 172.21.15.64:0/3894314512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00d0900 msgr2=0x7f49f00d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/3894314512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00d0900 0x7f49f00d0db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009e50 tx=0x7f49ec02bf80 comp rx=0 tx=0).stop 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 -- 172.21.15.64:0/3894314512 shutdown_connections 2024-04-01T00:14:28.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/3894314512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00d0900 0x7f49f00d0db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/3894314512 >> v2:172.21.15.100:3300/0 conn(0x7f49f0161a30 0x7f49f00d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 -- 172.21.15.64:0/3894314512 >> 172.21.15.64:0/3894314512 conn(0x7f49f009d250 msgr2=0x7f49f00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 -- 172.21.15.64:0/3894314512 shutdown_connections 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.291+0000 7f49f7ed4640 1 -- 172.21.15.64:0/3894314512 wait complete. 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.292+0000 7f49f7ed4640 1 Processor -- start 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.292+0000 7f49f7ed4640 1 -- start start 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f00d0900 0x7f49f01f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49f0161a30 0x7f49f015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00ceb40 con 0x7f49f00b3740 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00d2e00 con 0x7f49f00d0900 2024-04-01T00:14:28.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f7ed4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00cf3b0 con 0x7f49f0161a30 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f5c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f5c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47044/0 (socket says 172.21.15.64:47044) 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f5c70640 1 -- 172.21.15.64:0/2747310916 learned_addr learned my addr 172.21.15.64:0/2747310916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f546f640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f00d0900 0x7f49f01f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.293+0000 7f49f6471640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49f0161a30 0x7f49f015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 -- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49f0161a30 msgr2=0x7f49f015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49f0161a30 0x7f49f015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 -- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f00d0900 msgr2=0x7f49f01f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f00d0900 0x7f49f01f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 -- 172.21.15.64:0/2747310916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015b340 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f5c70640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e870 tx=0x7f49e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8017070 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e800c940 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f00b7d70 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800cd30 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.294+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f00b8330 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.295+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f00681a0 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.295+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f49e801a070 con 0x7f49f00b3740 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.296+0000 7f49e67fc640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 0x7f49d406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.296+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f49e808a720 con 0x7f49f00b3740 2024-04-01T00:14:28.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.296+0000 7f49f546f640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 0x7f49d406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.296+0000 7f49f546f640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 0x7f49d406de50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49f015d6e0 tx=0x7f49e0007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.300+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f49e8058c60 con 0x7f49f00b3740 2024-04-01T00:14:28.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.498+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49f009d730 con 0x7f49f00b3740 2024-04-01T00:14:28.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.498+0000 7f49e67fc640 1 -- 172.21.15.64:0/2747310916 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f49e805cd00 con 0x7f49f00b3740 2024-04-01T00:14:28.502 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 msgr2=0x7f49d406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 0x7f49d406de50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49f015d6e0 tx=0x7f49e0007b50 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 msgr2=0x7f49f015ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e870 tx=0x7f49e800ed30 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 shutdown_connections 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f49d406b990 0x7f49d406de50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49f0161a30 0x7f49f015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f00d0900 0x7f49f01f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 --2- 172.21.15.64:0/2747310916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f00b3740 0x7f49f015ae00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 >> 172.21.15.64:0/2747310916 conn(0x7f49f009d250 msgr2=0x7f49f00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 shutdown_connections 2024-04-01T00:14:28.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.500+0000 7f49f7ed4640 1 -- 172.21.15.64:0/2747310916 wait complete. 2024-04-01T00:14:28.512 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-04-01T00:14:28.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:28.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-01T00:14:28.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:28.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T00:14:28.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:28.539 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-01T00:14:28.594 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:28.594 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:28.610 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:28.610 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:28.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.767+0000 7f3d434d4640 1 Processor -- start 2024-04-01T00:14:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.767+0000 7f3d434d4640 1 -- start start 2024-04-01T00:14:28.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d3c15f610 0x7f3d3c070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c15f180 con 0x7f3d3c0cff40 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c0711e0 con 0x7f3d3c15f610 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d434d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c071460 con 0x7f3d3c151fe0 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47048/0 (socket says 172.21.15.64:47048) 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 -- 172.21.15.64:0/194564468 learned_addr learned my addr 172.21.15.64:0/194564468 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d40a6f640 1 --2- 172.21.15.64:0/194564468 >> v2:172.21.15.100:3300/0 conn(0x7f3d3c15f610 0x7f3d3c070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41270640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 -- 172.21.15.64:0/194564468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 msgr2=0x7f3d3c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 -- 172.21.15.64:0/194564468 >> v2:172.21.15.100:3300/0 conn(0x7f3d3c15f610 msgr2=0x7f3d3c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 --2- 172.21.15.64:0/194564468 >> v2:172.21.15.100:3300/0 conn(0x7f3d3c15f610 0x7f3d3c070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.768+0000 7f3d41a71640 1 -- 172.21.15.64:0/194564468 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c071740 con 0x7f3d3c0cff40 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d41a71640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0d0ac0 tx=0x7f3d3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b125fb23a48085a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d33fff640 1 -- 172.21.15.64:0/194564468 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3402ca70 con 0x7f3d3c0cff40 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d33fff640 1 -- 172.21.15.64:0/194564468 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3402cc10 con 0x7f3d3c0cff40 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d33fff640 1 -- 172.21.15.64:0/194564468 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340343f0 con 0x7f3d3c0cff40 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d434d4640 1 -- 172.21.15.64:0/194564468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 msgr2=0x7f3d3c0d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.769+0000 7f3d434d4640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0d0ac0 tx=0x7f3d3402bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 -- 172.21.15.64:0/194564468 shutdown_connections 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c0d03d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 --2- 172.21.15.64:0/194564468 >> v2:172.21.15.100:3300/0 conn(0x7f3d3c15f610 0x7f3d3c070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 --2- 172.21.15.64:0/194564468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 -- 172.21.15.64:0/194564468 >> 172.21.15.64:0/194564468 conn(0x7f3d3c06b4e0 msgr2=0x7f3d3c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 -- 172.21.15.64:0/194564468 shutdown_connections 2024-04-01T00:14:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.770+0000 7f3d434d4640 1 -- 172.21.15.64:0/194564468 wait complete. 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 Processor -- start 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 -- start start 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c09d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 0x7f3d3c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c0cda20 con 0x7f3d3c0cff40 2024-04-01T00:14:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c15ee10 con 0x7f3d3c15f610 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.771+0000 7f3d434d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c0cdba0 con 0x7f3d3c151fe0 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47058/0 (socket says 172.21.15.64:47058) 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 -- 172.21.15.64:0/2189648613 learned_addr learned my addr 172.21.15.64:0/2189648613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41a71640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 0x7f3d3c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d40a6f640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c09d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 -- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 msgr2=0x7f3d3c09d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c09d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 -- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 msgr2=0x7f3d3c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 0x7f3d3c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 -- 172.21.15.64:0/2189648613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c141a90 con 0x7f3d3c0cff40 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41a71640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 0x7f3d3c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d41270640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e9b0 tx=0x7f3d3800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38017070 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3800c9a0 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c09c070 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.772+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800cd90 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.773+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c09c630 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.773+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c0681a0 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.774+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3d3801a070 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.774+0000 7f3d31ffb640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 0x7f3d2006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.774+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3d3808a820 con 0x7f3d3c0cff40 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.774+0000 7f3d40a6f640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 0x7f3d2006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.775+0000 7f3d40a6f640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 0x7f3d2006de50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c144f90 tx=0x7f3d2c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:28.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.779+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3d3c141a90 con 0x7f3d3c0cff40 2024-04-01T00:14:28.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.964+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d3c15dba0 con 0x7f3d3c0cff40 2024-04-01T00:14:28.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.965+0000 7f3d31ffb640 1 -- 172.21.15.64:0/2189648613 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16830 (secure 0 0 0) 0x7f3d38058df0 con 0x7f3d3c0cff40 2024-04-01T00:14:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:28.115298+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 msgr2=0x7f3d2006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 0x7f3d2006de50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c144f90 tx=0x7f3d2c009190 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 msgr2=0x7f3d3c141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e9b0 tx=0x7f3d3800ee70 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 shutdown_connections 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3d2006b990 0x7f3d2006de50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3c15f610 0x7f3d3c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d3c151fe0 0x7f3d3c09d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 --2- 172.21.15.64:0/2189648613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d3c0cff40 0x7f3d3c141550 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 >> 172.21.15.64:0/2189648613 conn(0x7f3d3c06b4e0 msgr2=0x7f3d3c15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 shutdown_connections 2024-04-01T00:14:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:28.968+0000 7f3d434d4640 1 -- 172.21.15.64:0/2189648613 wait complete. 2024-04-01T00:14:28.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:28.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.084+0000 7f26a324e640 1 Processor -- start 2024-04-01T00:14:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.084+0000 7f26a324e640 1 -- start start 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a324e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 0x7f269c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a324e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c0b52c0 con 0x7f269c161a50 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a324e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c0b55a0 con 0x7f269c0d0f20 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a324e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0b5070 con 0x7f269c0b2df0 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a0fea640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f269c161a50 0x7f269c0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42440/0 (socket says 172.21.15.64:42440) 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f26a0fea640 1 -- 172.21.15.64:0/3842715771 learned_addr learned my addr 172.21.15.64:0/3842715771 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.085+0000 7f269bfff640 1 --2- 172.21.15.64:0/3842715771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 0x7f269c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486382355 0 0) 0x7f269c0b52c0 con 0x7f269c161a50 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2688003520 con 0x7f269c161a50 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781540749 0 0) 0x7f269c0b55a0 con 0x7f269c0d0f20 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f269c0b52c0 con 0x7f269c0d0f20 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269bfff640 1 -- 172.21.15.64:0/3842715771 >> v1:172.21.15.100:6789/0 conn(0x7f269c0d0f20 legacy=0x7f269c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269bfff640 1 -- 172.21.15.64:0/3842715771 >> v1:172.21.15.64:6789/0 conn(0x7f269c161a50 legacy=0x7f269c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.086+0000 7f269bfff640 1 -- 172.21.15.64:0/3842715771 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0d45a0 con 0x7f269c0b2df0 2024-04-01T00:14:29.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f269bfff640 1 --2- 172.21.15.64:0/3842715771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 0x7f269c0ceae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f268c009dc0 tx=0x7f268c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af49f1b3021137bc server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c039070 con 0x7f269c0b2df0 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c02cc40 con 0x7f269c0b2df0 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f269b7fe640 1 -- 172.21.15.64:0/3842715771 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c034440 con 0x7f269c0b2df0 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 -- 172.21.15.64:0/3842715771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 msgr2=0x7f269c0ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 --2- 172.21.15.64:0/3842715771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 0x7f269c0ceae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f268c009dc0 tx=0x7f268c02be90 comp rx=0 tx=0).stop 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 -- 172.21.15.64:0/3842715771 shutdown_connections 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 --2- 172.21.15.64:0/3842715771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c0b2df0 0x7f269c0ceae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 -- 172.21.15.64:0/3842715771 >> 172.21.15.64:0/3842715771 conn(0x7f269c09d1c0 msgr2=0x7f269c0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 -- 172.21.15.64:0/3842715771 shutdown_connections 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.087+0000 7f26a324e640 1 -- 172.21.15.64:0/3842715771 wait complete. 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.088+0000 7f26a324e640 1 Processor -- start 2024-04-01T00:14:29.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 -- start start 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269c0d0f20 0x7f269c0b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c161a50 0x7f269c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0b4950 con 0x7f269c0b2df0 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0d4bf0 con 0x7f269c0d0f20 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a324e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0b4ad0 con 0x7f269c161a50 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a0fea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a0fea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47072/0 (socket says 172.21.15.64:47072) 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.089+0000 7f26a0fea640 1 -- 172.21.15.64:0/740398563 learned_addr learned my addr 172.21.15.64:0/740398563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f269bfff640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269c0d0f20 0x7f269c0b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a17eb640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c161a50 0x7f269c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 -- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c161a50 msgr2=0x7f269c0b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c161a50 0x7f269c0b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 -- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269c0d0f20 msgr2=0x7f269c0b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269c0d0f20 0x7f269c0b93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 -- 172.21.15.64:0/740398563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0cc0a0 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a0fea640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f269400c530 tx=0x7f269400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694017070 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694005ce0 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694007270 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c0cc380 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.090+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c0b9910 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.091+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c0681a0 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.092+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f269401a070 con 0x7f269c0b2df0 2024-04-01T00:14:29.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.092+0000 7f26997fa640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 0x7f267c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.092+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2694089b70 con 0x7f269c0b2df0 2024-04-01T00:14:29.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.092+0000 7f269bfff640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 0x7f267c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.093+0000 7f269bfff640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 0x7f267c06de50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f268c036000 tx=0x7f268c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.096+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f26940580b0 con 0x7f269c0b2df0 2024-04-01T00:14:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.278+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f269c1607b0 con 0x7f269c0b2df0 2024-04-01T00:14:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.278+0000 7f26997fa640 1 -- 172.21.15.64:0/740398563 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f269405c150 con 0x7f269c0b2df0 2024-04-01T00:14:29.279 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:29.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 msgr2=0x7f267c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 0x7f267c06de50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f268c036000 tx=0x7f268c030000 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 msgr2=0x7f269c158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f269400c530 tx=0x7f269400c9f0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 shutdown_connections 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f267c06b990 0x7f267c06de50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269c161a50 0x7f269c0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269c0d0f20 0x7f269c0b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 --2- 172.21.15.64:0/740398563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f269c0b2df0 0x7f269c158d00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 >> 172.21.15.64:0/740398563 conn(0x7f269c09d1c0 msgr2=0x7f269c160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 shutdown_connections 2024-04-01T00:14:29.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.280+0000 7f26a324e640 1 -- 172.21.15.64:0/740398563 wait complete. 2024-04-01T00:14:29.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:29.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.395+0000 7f88bf3e5640 1 Processor -- start 2024-04-01T00:14:29.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 -- start start 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88b806ed00 0x7f88b806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b80cb020 con 0x7f88b8144720 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f88b8144b00 con 0x7f88b806ed00 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.396+0000 7f88bf3e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80cb300 con 0x7f88b80cab60 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bd181640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f88b8144720 0x7f88b806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42444/0 (socket says 172.21.15.64:42444) 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bd181640 1 -- 172.21.15.64:0/3089148692 learned_addr learned my addr 172.21.15.64:0/3089148692 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bc980640 1 --2- 172.21.15.64:0/3089148692 >> v2:172.21.15.100:3300/0 conn(0x7f88b806ed00 0x7f88b806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bd982640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88affff640 1 -- 172.21.15.64:0/3089148692 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44125485 0 0) 0x7f88b80cb020 con 0x7f88b8144720 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88affff640 1 -- 172.21.15.64:0/3089148692 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a4003520 con 0x7f88b8144720 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bc980640 1 -- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 msgr2=0x7f88b8070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bc980640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bc980640 1 -- 172.21.15.64:0/3089148692 >> v1:172.21.15.64:6789/0 conn(0x7f88b8144720 legacy=0x7f88b806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bc980640 1 -- 172.21.15.64:0/3089148692 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80cb5e0 con 0x7f88b806ed00 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.397+0000 7f88bd982640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88bc980640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b806ed00 0x7f88b806ce70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc633fce7e5cdea server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88affff640 1 -- 172.21.15.64:0/3089148692 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8035020 con 0x7f88b806ed00 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88affff640 1 -- 172.21.15.64:0/3089148692 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a8031830 con 0x7f88b806ed00 2024-04-01T00:14:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88affff640 1 -- 172.21.15.64:0/3089148692 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8031c00 con 0x7f88b806ed00 2024-04-01T00:14:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b806ed00 msgr2=0x7f88b806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.398+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b806ed00 0x7f88b806ce70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3089148692 shutdown_connections 2024-04-01T00:14:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3089148692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b806ed00 0x7f88b806ce70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3089148692 >> 172.21.15.64:0/3089148692 conn(0x7f88b80cfa90 msgr2=0x7f88b80cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3089148692 shutdown_connections 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.399+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3089148692 wait complete. 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 Processor -- start 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 -- start start 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b8144720 0x7f88b80b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80cd7f0 con 0x7f88b806ed00 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80cbba0 con 0x7f88b8144720 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.400+0000 7f88bf3e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80cd970 con 0x7f88b80cab60 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47078/0 (socket says 172.21.15.64:47078) 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 -- 172.21.15.64:0/3329743355 learned_addr learned my addr 172.21.15.64:0/3329743355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd982640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b8144720 0x7f88b80b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bc980640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 -- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 msgr2=0x7f88b8140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 -- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b8144720 msgr2=0x7f88b80b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b8144720 0x7f88b80b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 -- 172.21.15.64:0/3329743355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b8137cc0 con 0x7f88b806ed00 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bd181640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c390 tx=0x7f88b400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4017070 con 0x7f88b806ed00 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b4005ce0 con 0x7f88b806ed00 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b8137420 con 0x7f88b806ed00 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40072f0 con 0x7f88b806ed00 2024-04-01T00:14:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.401+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81379e0 con 0x7f88b806ed00 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.402+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b80681a0 con 0x7f88b806ed00 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.403+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f88b401a070 con 0x7f88b806ed00 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.403+0000 7f88adffb640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 0x7f889c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.403+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f88b4089b50 con 0x7f88b806ed00 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.403+0000 7f88bc980640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 0x7f889c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.404+0000 7f88bc980640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 0x7f889c06de80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88a8036000 tx=0x7f88a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.407+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f88b4058120 con 0x7f88b806ed00 2024-04-01T00:14:29.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.588+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88b80cf230 con 0x7f88b806ed00 2024-04-01T00:14:29.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.588+0000 7f88adffb640 1 -- 172.21.15.64:0/3329743355 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f88b405c1c0 con 0x7f88b806ed00 2024-04-01T00:14:29.589 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 msgr2=0x7f889c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 0x7f889c06de80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88a8036000 tx=0x7f88a8008030 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 msgr2=0x7f88b80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c390 tx=0x7f88b400c850 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 shutdown_connections 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f889c06b9c0 0x7f889c06de80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b8144720 0x7f88b80b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88b80cab60 0x7f88b8140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 --2- 172.21.15.64:0/3329743355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b806ed00 0x7f88b80b27b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.590+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 >> 172.21.15.64:0/3329743355 conn(0x7f88b80cfa90 msgr2=0x7f88b8137f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.591+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 shutdown_connections 2024-04-01T00:14:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.591+0000 7f88bf3e5640 1 -- 172.21.15.64:0/3329743355 wait complete. 2024-04-01T00:14:29.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:29.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.706+0000 7f1ff295c640 1 Processor -- start 2024-04-01T00:14:29.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.706+0000 7f1ff295c640 1 -- start start 2024-04-01T00:14:29.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff295c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff295c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 0x7f1fec0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff295c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fec15c7a0 con 0x7f1fec15c3c0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff295c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f1fec0d2be0 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff295c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec0cdb00 con 0x7f1fec0c6970 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47088/0 (socket says 172.21.15.64:47088) 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 -- 172.21.15.64:0/872330394 learned_addr learned my addr 172.21.15.64:0/872330394 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff0ef9640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 0x7f1fec0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 -- 172.21.15.64:0/872330394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 msgr2=0x7f1fec0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 0x7f1fec0cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 -- 172.21.15.64:0/872330394 >> v1:172.21.15.100:6789/0 conn(0x7f1fec15c3c0 legacy=0x7f1fec0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feb7fe640 1 -- 172.21.15.64:0/872330394 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0d2900 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1feaffd640 1 -- 172.21.15.64:0/872330394 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935514426 0 0) 0x7f1fec15c7a0 con 0x7f1fec15c3c0 2024-04-01T00:14:29.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.707+0000 7f1ff0ef9640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 0x7f1fec0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.708+0000 7f1feb7fe640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009870 tx=0x7f1fe002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4220f777fefb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.708+0000 7f1feaffd640 1 -- 172.21.15.64:0/872330394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00317c0 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.708+0000 7f1feaffd640 1 -- 172.21.15.64:0/872330394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0037070 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.708+0000 7f1feaffd640 1 -- 172.21.15.64:0/872330394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0031b80 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 -- 172.21.15.64:0/872330394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec0ceaa0 msgr2=0x7f1fec0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009870 tx=0x7f1fe002b860 comp rx=0 tx=0).stop 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 -- 172.21.15.64:0/872330394 shutdown_connections 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec0ceaa0 0x7f1fec0cfa40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 --2- 172.21.15.64:0/872330394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0c6970 0x7f1fec0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 -- 172.21.15.64:0/872330394 >> 172.21.15.64:0/872330394 conn(0x7f1fec0d1c60 msgr2=0x7f1fec0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 -- 172.21.15.64:0/872330394 shutdown_connections 2024-04-01T00:14:29.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.709+0000 7f1ff295c640 1 -- 172.21.15.64:0/872330394 wait complete. 2024-04-01T00:14:29.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.710+0000 7f1ff295c640 1 Processor -- start 2024-04-01T00:14:29.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.710+0000 7f1ff295c640 1 -- start start 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec0c6970 0x7f1fec0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0ceaa0 0x7f1fec09f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec0cef50 con 0x7f1fec15c3c0 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec0cd790 con 0x7f1fec0c6970 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff295c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec0b2d70 con 0x7f1fec0ceaa0 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1feb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0ceaa0 0x7f1fec09f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47104/0 (socket says 172.21.15.64:47104) 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1febfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec0c6970 0x7f1fec0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 -- 172.21.15.64:0/2377762061 learned_addr learned my addr 172.21.15.64:0/2377762061 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 -- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0ceaa0 msgr2=0x7f1fec09f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0ceaa0 0x7f1fec09f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 -- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec0c6970 msgr2=0x7f1fec0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec0c6970 0x7f1fec0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 -- 172.21.15.64:0/2377762061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0c9d80 con 0x7f1fec15c3c0 2024-04-01T00:14:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1ff0ef9640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400efc0 tx=0x7f1fe4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.711+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4018020 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.712+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe4014480 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.712+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec0ca040 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.712+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4014870 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.712+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec0ca600 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.712+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec0681a0 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.713+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1fe4005d10 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.713+0000 7f1fe8ff9640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 0x7f1fd806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.713+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1fe408b570 con 0x7f1fec15c3c0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.713+0000 7f1febfff640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 0x7f1fd806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.714+0000 7f1febfff640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 0x7f1fd806de50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:29.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.718+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f1fe4022030 con 0x7f1fec15c3c0 2024-04-01T00:14:29.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.900+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1fec0cbb40 con 0x7f1fec15c3c0 2024-04-01T00:14:29.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.900+0000 7f1fe8ff9640 1 -- 172.21.15.64:0/2377762061 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f1fe4010040 con 0x7f1fec15c3c0 2024-04-01T00:14:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:29.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 msgr2=0x7f1fd806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 0x7f1fd806de50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).stop 2024-04-01T00:14:29.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 msgr2=0x7f1fec0d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400efc0 tx=0x7f1fe4009f90 comp rx=0 tx=0).stop 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 shutdown_connections 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1fd806b990 0x7f1fd806de50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fec15c3c0 0x7f1fec0d09c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fec0ceaa0 0x7f1fec09f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 --2- 172.21.15.64:0/2377762061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec0c6970 0x7f1fec0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 >> 172.21.15.64:0/2377762061 conn(0x7f1fec0d1c60 msgr2=0x7f1fec0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 shutdown_connections 2024-04-01T00:14:29.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:29.902+0000 7f1ff295c640 1 -- 172.21.15.64:0/2377762061 wait complete. 2024-04-01T00:14:29.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:30.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.016+0000 7f35635dd640 1 Processor -- start 2024-04-01T00:14:30.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 -- start start 2024-04-01T00:14:30.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f355c0b5670 0x7f355c0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d23b0 0x7f355c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f355c0d0020 con 0x7f355c0d0d90 2024-04-01T00:14:30.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f355c0cbc80 con 0x7f355c0b5670 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.017+0000 7f35635dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f355c0d0300 con 0x7f355c0d23b0 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f355c0b5670 0x7f355c0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f355c0b5670 0x7f355c0d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47118/0 (socket says 172.21.15.64:47118) 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 -- 172.21.15.64:0/3639495829 learned_addr learned my addr 172.21.15.64:0/3639495829 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3560b78640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d23b0 0x7f355c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 -- 172.21.15.64:0/3639495829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d23b0 msgr2=0x7f355c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d23b0 0x7f355c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 -- 172.21.15.64:0/3639495829 >> v1:172.21.15.100:6789/0 conn(0x7f355c0d0d90 legacy=0x7f355c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.018+0000 7f3561379640 1 -- 172.21.15.64:0/3639495829 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c0d2dc0 con 0x7f355c0b5670 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f3561379640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c0b5670 0x7f355c0d2880 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3558009870 tx=0x7f355802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4236fc234719bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f3553fff640 1 -- 172.21.15.64:0/3639495829 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558034070 con 0x7f355c0b5670 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f3553fff640 1 -- 172.21.15.64:0/3639495829 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3558030840 con 0x7f355c0b5670 2024-04-01T00:14:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f3553fff640 1 -- 172.21.15.64:0/3639495829 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558030c10 con 0x7f355c0b5670 2024-04-01T00:14:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 -- 172.21.15.64:0/3639495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c0b5670 msgr2=0x7f355c0d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c0b5670 0x7f355c0d2880 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3558009870 tx=0x7f355802b980 comp rx=0 tx=0).stop 2024-04-01T00:14:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 -- 172.21.15.64:0/3639495829 shutdown_connections 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d23b0 0x7f355c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 --2- 172.21.15.64:0/3639495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c0b5670 0x7f355c0d2880 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 -- 172.21.15.64:0/3639495829 >> 172.21.15.64:0/3639495829 conn(0x7f355c09d250 msgr2=0x7f355c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 -- 172.21.15.64:0/3639495829 shutdown_connections 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.019+0000 7f35635dd640 1 -- 172.21.15.64:0/3639495829 wait complete. 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.020+0000 7f35635dd640 1 Processor -- start 2024-04-01T00:14:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.020+0000 7f35635dd640 1 -- start start 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d0d90 0x7f355c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c0d23b0 0x7f355c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0cf340 con 0x7f355c157570 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0cd330 con 0x7f355c0d23b0 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f35635dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f355c0cf4c0 con 0x7f355c0d0d90 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f3561b7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f3561b7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47124/0 (socket says 172.21.15.64:47124) 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f3561379640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d0d90 0x7f355c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f3561b7a640 1 -- 172.21.15.64:0/879049669 learned_addr learned my addr 172.21.15.64:0/879049669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.021+0000 7f3560b78640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c0d23b0 0x7f355c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 -- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d0d90 msgr2=0x7f355c1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d0d90 0x7f355c1f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 -- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c0d23b0 msgr2=0x7f355c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c0d23b0 0x7f355c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 -- 172.21.15.64:0/879049669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c15b5d0 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3561b7a640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f355400c7a0 tx=0x7f355400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554003b70 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3554003d30 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c0c8ee0 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400aa90 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.022+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c151890 con 0x7f355c157570 2024-04-01T00:14:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.023+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c0681a0 con 0x7f355c157570 2024-04-01T00:14:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.023+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f355400ac30 con 0x7f355c157570 2024-04-01T00:14:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.024+0000 7f3551ffb640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 0x7f354006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.024+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f355408a630 con 0x7f355c157570 2024-04-01T00:14:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.024+0000 7f3561379640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 0x7f354006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.024+0000 7f3561379640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 0x7f354006de50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3558036000 tx=0x7f3558008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.028+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3554058c00 con 0x7f355c157570 2024-04-01T00:14:30.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.220+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f355c0c7690 con 0x7f355c157570 2024-04-01T00:14:30.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.220+0000 7f3551ffb640 1 -- 172.21.15.64:0/879049669 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f355405cca0 con 0x7f355c157570 2024-04-01T00:14:30.221 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:30.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 msgr2=0x7f354006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 0x7f354006de50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3558036000 tx=0x7f3558008030 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 msgr2=0x7f355c151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f355400c7a0 tx=0x7f355400cc60 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 shutdown_connections 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f354006b990 0x7f354006de50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f355c157570 0x7f355c151350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c0d23b0 0x7f355c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 --2- 172.21.15.64:0/879049669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f355c0d0d90 0x7f355c1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 >> 172.21.15.64:0/879049669 conn(0x7f355c09d250 msgr2=0x7f355c06ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 shutdown_connections 2024-04-01T00:14:30.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.222+0000 7f35635dd640 1 -- 172.21.15.64:0/879049669 wait complete. 2024-04-01T00:14:30.234 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T00:14:30.234 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T00:14:30.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:30.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.352+0000 7ff88736c640 1 Processor -- start 2024-04-01T00:14:30.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.352+0000 7ff88736c640 1 -- start start 2024-04-01T00:14:30.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.352+0000 7ff88736c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff880151fe0 0x7ff88015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff88736c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff88015f610 0x7ff880070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff88736c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8800711e0 con 0x7ff8800cff40 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff88736c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7ff88015f180 con 0x7ff88015f610 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff88736c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff880071460 con 0x7ff880151fe0 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff88015f610 0x7ff880070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff88015f610 0x7ff880070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47132/0 (socket says 172.21.15.64:47132) 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 -- 172.21.15.64:0/3474215714 learned_addr learned my addr 172.21.15.64:0/3474215714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff885108640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff880151fe0 0x7ff88015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 -- 172.21.15.64:0/3474215714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff880151fe0 msgr2=0x7ff88015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff880151fe0 0x7ff88015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 -- 172.21.15.64:0/3474215714 >> v1:172.21.15.100:6789/0 conn(0x7ff8800cff40 legacy=0x7ff8800cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.353+0000 7ff884907640 1 -- 172.21.15.64:0/3474215714 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff880071740 con 0x7ff88015f610 2024-04-01T00:14:30.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff884907640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff88015f610 0x7ff880070ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff874009870 tx=0x7ff87402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b7dcbd051ae1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff873fff640 1 -- 172.21.15.64:0/3474215714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874034070 con 0x7ff88015f610 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff873fff640 1 -- 172.21.15.64:0/3474215714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff874030840 con 0x7ff88015f610 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff873fff640 1 -- 172.21.15.64:0/3474215714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874030c10 con 0x7ff88015f610 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 -- 172.21.15.64:0/3474215714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff88015f610 msgr2=0x7ff880070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff88015f610 0x7ff880070ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff874009870 tx=0x7ff87402b980 comp rx=0 tx=0).stop 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 -- 172.21.15.64:0/3474215714 shutdown_connections 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff88015f610 0x7ff880070ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474215714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff880151fe0 0x7ff88015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 -- 172.21.15.64:0/3474215714 >> 172.21.15.64:0/3474215714 conn(0x7ff88006b4e0 msgr2=0x7ff88006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.354+0000 7ff88736c640 1 -- 172.21.15.64:0/3474215714 shutdown_connections 2024-04-01T00:14:30.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.355+0000 7ff88736c640 1 -- 172.21.15.64:0/3474215714 wait complete. 2024-04-01T00:14:30.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 Processor -- start 2024-04-01T00:14:30.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 -- start start 2024-04-01T00:14:30.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8800cff40 0x7ff880150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88015f610 0x7ff8800b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8800d03d0 con 0x7ff880151fe0 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff88015ee10 con 0x7ff88015f610 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff88736c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff8800d0550 con 0x7ff8800cff40 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff884907640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff884907640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47134/0 (socket says 172.21.15.64:47134) 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff884907640 1 -- 172.21.15.64:0/3474494506 learned_addr learned my addr 172.21.15.64:0/3474494506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff885108640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8800cff40 0x7ff880150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.356+0000 7ff885909640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88015f610 0x7ff8800b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 -- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8800cff40 msgr2=0x7ff880150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8800cff40 0x7ff880150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 -- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88015f610 msgr2=0x7ff8800b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88015f610 0x7ff8800b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 -- 172.21.15.64:0/3474494506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff880150ac0 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff884907640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff874000c00 tx=0x7ff8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874034040 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff874039d90 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8800acf00 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874043630 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.357+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8800ad4c0 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.358+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8800681a0 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.359+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff87404a060 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.359+0000 7ff871ffb640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 0x7ff86406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.359+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff8740ac640 con 0x7ff880151fe0 2024-04-01T00:14:30.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.359+0000 7ff885108640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 0x7ff86406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.360+0000 7ff885108640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 0x7ff86406de50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff88015d9e0 tx=0x7ff87c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.363+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff87403e070 con 0x7ff880151fe0 2024-04-01T00:14:30.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.548+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff88015be40 con 0x7ff880151fe0 2024-04-01T00:14:30.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.549+0000 7ff871ffb640 1 -- 172.21.15.64:0/3474494506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16830 (secure 0 0 0) 0x7ff87407ac10 con 0x7ff880151fe0 2024-04-01T00:14:30.551 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:28.115298+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 msgr2=0x7ff86406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 0x7ff86406de50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff88015d9e0 tx=0x7ff87c009190 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 msgr2=0x7ff880147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff874000c00 tx=0x7ff8740046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 shutdown_connections 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff86406b990 0x7ff86406de50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88015f610 0x7ff8800b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff880151fe0 0x7ff880147540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 --2- 172.21.15.64:0/3474494506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8800cff40 0x7ff880150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.551+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 >> 172.21.15.64:0/3474494506 conn(0x7ff88006b4e0 msgr2=0x7ff8800cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.552+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 shutdown_connections 2024-04-01T00:14:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.552+0000 7ff88736c640 1 -- 172.21.15.64:0/3474494506 wait complete. 2024-04-01T00:14:30.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:30.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:30.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 Processor -- start 2024-04-01T00:14:30.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 -- start start 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b1850 0x7f4be00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4be00cf740 0x7f4be00cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00d00f0 con 0x7f4be00b0ba0 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4be00b0f80 con 0x7f4be00cf740 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.668+0000 7f4be66eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00cc730 con 0x7f4be00b1850 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4be00cf740 0x7f4be00cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4bdffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4be00b0ba0 0x7f4be00b1140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51910/0 (socket says 172.21.15.64:51910) 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b1850 0x7f4be00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 --2- 172.21.15.64:0/3047990002 >> v2:172.21.15.100:3300/0 conn(0x7f4be00cf740 0x7f4be00cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:35768/0 (socket says 172.21.15.64:35768) 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4bdffff640 1 -- 172.21.15.64:0/3047990002 learned_addr learned my addr 172.21.15.64:0/3047990002 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:30.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 -- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b1850 msgr2=0x7f4be00cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:30.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 --2- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b1850 0x7f4be00cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 -- 172.21.15.64:0/3047990002 >> v1:172.21.15.100:6789/0 conn(0x7f4be00b0ba0 legacy=0x7f4be00b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.669+0000 7f4be4c88640 1 -- 172.21.15.64:0/3047990002 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00cca10 con 0x7f4be00cf740 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be4c88640 1 --2- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00cf740 0x7f4be00cfbb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009d90 tx=0x7f4bd802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e5417e0ec88c374 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4bdeffd640 1 -- 172.21.15.64:0/3047990002 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8039070 con 0x7f4be00cf740 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4bdeffd640 1 -- 172.21.15.64:0/3047990002 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd802cc50 con 0x7f4be00cf740 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4bdeffd640 1 -- 172.21.15.64:0/3047990002 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8034440 con 0x7f4be00cf740 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 -- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00cf740 msgr2=0x7f4be00cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 --2- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00cf740 0x7f4be00cfbb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009d90 tx=0x7f4bd802bea0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 -- 172.21.15.64:0/3047990002 shutdown_connections 2024-04-01T00:14:30.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 --2- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00cf740 0x7f4be00cfbb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 --2- 172.21.15.64:0/3047990002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b1850 0x7f4be00cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 -- 172.21.15.64:0/3047990002 >> 172.21.15.64:0/3047990002 conn(0x7f4be0072810 msgr2=0x7f4be0072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 -- 172.21.15.64:0/3047990002 shutdown_connections 2024-04-01T00:14:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.670+0000 7f4be66eb640 1 -- 172.21.15.64:0/3047990002 wait complete. 2024-04-01T00:14:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.672+0000 7f4be66eb640 1 Processor -- start 2024-04-01T00:14:30.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.672+0000 7f4be66eb640 1 -- start start 2024-04-01T00:14:30.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.672+0000 7f4be66eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b0ba0 0x7f4be00c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.672+0000 7f4be66eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00b1850 0x7f4be00b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00cd2b0 con 0x7f4be00cf740 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00ccfd0 con 0x7f4be00b1850 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4be0162e10 con 0x7f4be00b0ba0 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47150/0 (socket says 172.21.15.64:47150) 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 -- 172.21.15.64:0/4192740102 learned_addr learned my addr 172.21.15.64:0/4192740102 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4bdffff640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b0ba0 0x7f4be00c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4bdf7fe640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00b1850 0x7f4be00b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 -- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b0ba0 msgr2=0x7f4be00c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b0ba0 0x7f4be00c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 -- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00b1850 msgr2=0x7f4be00b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00b1850 0x7f4be00b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 -- 172.21.15.64:0/4192740102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00c58d0 con 0x7f4be00cf740 2024-04-01T00:14:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be4c88640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009d90 tx=0x7f4bd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8039050 con 0x7f4be00cf740 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd8045070 con 0x7f4be00cf740 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd803e610 con 0x7f4be00cf740 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be00c5bb0 con 0x7f4be00cf740 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.673+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be00c6170 con 0x7f4be00cf740 2024-04-01T00:14:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.675+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be00681a0 con 0x7f4be00cf740 2024-04-01T00:14:30.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.675+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f4bd803e7b0 con 0x7f4be00cf740 2024-04-01T00:14:30.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.675+0000 7f4bdcff9640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 0x7f4bcc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.675+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4bd80ad280 con 0x7f4be00cf740 2024-04-01T00:14:30.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.675+0000 7f4bdffff640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 0x7f4bcc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.676+0000 7f4bdffff640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 0x7f4bcc06de50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.680+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4bd807b7d0 con 0x7f4be00cf740 2024-04-01T00:14:30.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.864+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4be0162610 con 0x7f4be00cf740 2024-04-01T00:14:30.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.865+0000 7f4bdcff9640 1 -- 172.21.15.64:0/4192740102 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4bd807f870 con 0x7f4be00cf740 2024-04-01T00:14:30.867 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 msgr2=0x7f4bcc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 0x7f4bcc06de50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0007e80 comp rx=0 tx=0).stop 2024-04-01T00:14:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 msgr2=0x7f4be00c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009d90 tx=0x7f4bd8004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 shutdown_connections 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4bcc06b990 0x7f4bcc06de50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00cf740 0x7f4be00c9530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00b1850 0x7f4be00b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 --2- 172.21.15.64:0/4192740102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00b0ba0 0x7f4be00c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 >> 172.21.15.64:0/4192740102 conn(0x7f4be0072810 msgr2=0x7f4be006d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 shutdown_connections 2024-04-01T00:14:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.867+0000 7f4be66eb640 1 -- 172.21.15.64:0/4192740102 wait complete. 2024-04-01T00:14:30.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.983+0000 7fae0af02640 1 Processor -- start 2024-04-01T00:14:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.983+0000 7fae0af02640 1 -- start start 2024-04-01T00:14:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0af02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ca600 0x7fae040d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0af02640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae040d0f20 0x7fae040cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0af02640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae04161d50 con 0x7fae040d4b70 2024-04-01T00:14:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0af02640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fae040b5020 con 0x7fae040d0f20 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0af02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fae040b4b40 con 0x7fae040ca600 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae040d0f20 0x7fae040cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae040d0f20 0x7fae040cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47166/0 (socket says 172.21.15.64:47166) 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 -- 172.21.15.64:0/3152889743 learned_addr learned my addr 172.21.15.64:0/3152889743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae03fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fae040d4b70 0x7fae040c79f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51918/0 (socket says 172.21.15.64:51918) 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae08c9e640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ca600 0x7fae040d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 -- 172.21.15.64:0/3152889743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ca600 msgr2=0x7fae040d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae037fe640 1 -- 172.21.15.64:0/3152889743 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42271058 0 0) 0x7fae04161d50 con 0x7fae040d4b70 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ca600 0x7fae040d2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 -- 172.21.15.64:0/3152889743 >> v1:172.21.15.100:6789/0 conn(0x7fae040d4b70 legacy=0x7fae040c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.984+0000 7fae0949f640 1 -- 172.21.15.64:0/3152889743 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae040d1e90 con 0x7fae040d0f20 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.985+0000 7fae0949f640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040d0f20 0x7fae040cfa50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1140a97313672797 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.985+0000 7fae037fe640 1 -- 172.21.15.64:0/3152889743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec034070 con 0x7fae040d0f20 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.985+0000 7fae037fe640 1 -- 172.21.15.64:0/3152889743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec030840 con 0x7fae040d0f20 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.985+0000 7fae037fe640 1 -- 172.21.15.64:0/3152889743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec030c10 con 0x7fae040d0f20 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 -- 172.21.15.64:0/3152889743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040d0f20 msgr2=0x7fae040cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040d0f20 0x7fae040cfa50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b980 comp rx=0 tx=0).stop 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 -- 172.21.15.64:0/3152889743 shutdown_connections 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040d0f20 0x7fae040cfa50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 --2- 172.21.15.64:0/3152889743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ca600 0x7fae040d2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 -- 172.21.15.64:0/3152889743 >> 172.21.15.64:0/3152889743 conn(0x7fae040d3ea0 msgr2=0x7fae040d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 -- 172.21.15.64:0/3152889743 shutdown_connections 2024-04-01T00:14:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.986+0000 7fae0af02640 1 -- 172.21.15.64:0/3152889743 wait complete. 2024-04-01T00:14:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.987+0000 7fae0af02640 1 Processor -- start 2024-04-01T00:14:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.987+0000 7fae0af02640 1 -- start start 2024-04-01T00:14:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae040d0f20 0x7fae04151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040d4b70 0x7fae04152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae040d2b40 con 0x7fae040ca600 2024-04-01T00:14:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae040d29c0 con 0x7fae040d0f20 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0af02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fae040d2cc0 con 0x7fae040d4b70 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47182/0 (socket says 172.21.15.64:47182) 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 -- 172.21.15.64:0/3278039299 learned_addr learned my addr 172.21.15.64:0/3278039299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae0949f640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040d4b70 0x7fae04152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae03fff640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae040d0f20 0x7fae04151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 -- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040d4b70 msgr2=0x7fae04152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040d4b70 0x7fae04152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 -- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae040d0f20 msgr2=0x7fae04151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae040d0f20 0x7fae04151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 -- 172.21.15.64:0/3278039299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae040b1710 con 0x7fae040ca600 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.988+0000 7fae08c9e640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00e9b0 tx=0x7fadfc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.989+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc017070 con 0x7fae040ca600 2024-04-01T00:14:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.989+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc00c9a0 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.989+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00cd90 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.989+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae040b19f0 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.989+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae040b1fb0 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.990+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae040681a0 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.990+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fadfc01a070 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.991+0000 7fae017fa640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 0x7fade406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.991+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fadfc08ab70 con 0x7fae040ca600 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.991+0000 7fae03fff640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 0x7fade406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.991+0000 7fae03fff640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 0x7fade406de50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae040b67b0 tx=0x7fadf4001630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:30.995+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fadfc059140 con 0x7fae040ca600 2024-04-01T00:14:31.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.177+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae040d2260 con 0x7fae040ca600 2024-04-01T00:14:31.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.177+0000 7fae017fa640 1 -- 172.21.15.64:0/3278039299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fadfc05d1e0 con 0x7fae040ca600 2024-04-01T00:14:31.178 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 msgr2=0x7fade406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 0x7fade406de50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae040b67b0 tx=0x7fadf4001630 comp rx=0 tx=0).stop 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 msgr2=0x7fae0406c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00e9b0 tx=0x7fadfc00ee70 comp rx=0 tx=0).stop 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 shutdown_connections 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fade406b990 0x7fade406de50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040d4b70 0x7fae04152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae040d0f20 0x7fae04151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 --2- 172.21.15.64:0/3278039299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae040ca600 0x7fae0406c260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 >> 172.21.15.64:0/3278039299 conn(0x7fae040d3ea0 msgr2=0x7fae040c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 shutdown_connections 2024-04-01T00:14:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.179+0000 7fae0af02640 1 -- 172.21.15.64:0/3278039299 wait complete. 2024-04-01T00:14:31.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.295+0000 7ff7be62e640 1 Processor -- start 2024-04-01T00:14:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.295+0000 7ff7be62e640 1 -- start start 2024-04-01T00:14:31.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7be62e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 0x7ff7b80c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7be62e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 0x7ff7b8161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7be62e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b8144ae0 con 0x7ff7b8162290 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7be62e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80cb200 con 0x7ff7b80c5fb0 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7be62e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80c6410 con 0x7ff7b8144700 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7bcbcb640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff7b8162290 0x7ff7b8071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42458/0 (socket says 172.21.15.64:42458) 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7bcbcb640 1 -- 172.21.15.64:0/254954465 learned_addr learned my addr 172.21.15.64:0/254954465 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b7fff640 1 --2- 172.21.15.64:0/254954465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 0x7ff7b80c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b77fe640 1 --2- 172.21.15.64:0/254954465 >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 0x7ff7b8161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b6ffd640 1 -- 172.21.15.64:0/254954465 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983432361 0 0) 0x7ff7b8144ae0 con 0x7ff7b8162290 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b6ffd640 1 -- 172.21.15.64:0/254954465 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7a0003520 con 0x7ff7b8162290 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b7fff640 1 -- 172.21.15.64:0/254954465 >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 msgr2=0x7ff7b8161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b7fff640 1 --2- 172.21.15.64:0/254954465 >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 0x7ff7b8161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b7fff640 1 -- 172.21.15.64:0/254954465 >> v1:172.21.15.64:6789/0 conn(0x7ff7b8162290 legacy=0x7ff7b8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b7fff640 1 -- 172.21.15.64:0/254954465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8162af0 con 0x7ff7b8144700 2024-04-01T00:14:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.296+0000 7ff7b77fe640 1 --2- 172.21.15.64:0/254954465 >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 0x7ff7b8161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.297+0000 7ff7b7fff640 1 --2- 172.21.15.64:0/254954465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 0x7ff7b80c5a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904806188ff2ac1f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.297+0000 7ff7b6ffd640 1 -- 172.21.15.64:0/254954465 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80315a0 con 0x7ff7b8144700 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.297+0000 7ff7b6ffd640 1 -- 172.21.15.64:0/254954465 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8037070 con 0x7ff7b8144700 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.297+0000 7ff7b6ffd640 1 -- 172.21.15.64:0/254954465 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8031960 con 0x7ff7b8144700 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 -- 172.21.15.64:0/254954465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 msgr2=0x7ff7b80c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 --2- 172.21.15.64:0/254954465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 0x7ff7b80c5a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b840 comp rx=0 tx=0).stop 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 -- 172.21.15.64:0/254954465 shutdown_connections 2024-04-01T00:14:31.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 --2- 172.21.15.64:0/254954465 >> v2:172.21.15.100:3300/0 conn(0x7ff7b80c5fb0 0x7ff7b8161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 --2- 172.21.15.64:0/254954465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b8144700 0x7ff7b80c5a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 -- 172.21.15.64:0/254954465 >> 172.21.15.64:0/254954465 conn(0x7ff7b80d0bb0 msgr2=0x7ff7b80d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 -- 172.21.15.64:0/254954465 shutdown_connections 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.298+0000 7ff7be62e640 1 -- 172.21.15.64:0/254954465 wait complete. 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.299+0000 7ff7be62e640 1 Processor -- start 2024-04-01T00:14:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.299+0000 7ff7be62e640 1 -- start start 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b80c5fb0 0x7ff7b815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8144700 0x7ff7b80c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80cf070 con 0x7ff7b8162290 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80ced90 con 0x7ff7b8144700 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7be62e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80d0880 con 0x7ff7b80c5fb0 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b80c5fb0 0x7ff7b815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8144700 0x7ff7b80c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47188/0 (socket says 172.21.15.64:47188) 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 -- 172.21.15.64:0/1690808718 learned_addr learned my addr 172.21.15.64:0/1690808718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 -- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b80c5fb0 msgr2=0x7ff7b815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b80c5fb0 0x7ff7b815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 -- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8144700 msgr2=0x7ff7b80c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8144700 0x7ff7b80c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 -- 172.21.15.64:0/1690808718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80c96b0 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.300+0000 7ff7bcbcb640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cb40 tx=0x7ff7b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.301+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0017070 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.301+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b000aca0 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.301+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b80c9990 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.301+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0012620 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.301+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b80c9f50 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.302+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b80681a0 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.302+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff7b001a070 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.302+0000 7ff7b4ff9640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 0x7ff7a406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.303+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff7b008a710 con 0x7ff7b8162290 2024-04-01T00:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.303+0000 7ff7b7fff640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 0x7ff7a406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.303+0000 7ff7b7fff640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 0x7ff7a406de50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8036000 tx=0x7ff7a803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.307+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff7b0058bd0 con 0x7ff7b8162290 2024-04-01T00:14:31.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.491+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7b8072be0 con 0x7ff7b8162290 2024-04-01T00:14:31.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.491+0000 7ff7b4ff9640 1 -- 172.21.15.64:0/1690808718 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff7b005cc70 con 0x7ff7b8162290 2024-04-01T00:14:31.492 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:31.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 msgr2=0x7ff7a406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 0x7ff7a406de50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8036000 tx=0x7ff7a803d040 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 msgr2=0x7ff7b815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cb40 tx=0x7ff7b000a060 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 shutdown_connections 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff7a406b990 0x7ff7a406de50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b8162290 0x7ff7b815e6b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8144700 0x7ff7b80c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 --2- 172.21.15.64:0/1690808718 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b80c5fb0 0x7ff7b815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 >> 172.21.15.64:0/1690808718 conn(0x7ff7b80d0bb0 msgr2=0x7ff7b8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 shutdown_connections 2024-04-01T00:14:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.493+0000 7ff7be62e640 1 -- 172.21.15.64:0/1690808718 wait complete. 2024-04-01T00:14:31.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-01T00:14:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.607+0000 7ffb1b5c9640 1 Processor -- start 2024-04-01T00:14:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.607+0000 7ffb1b5c9640 1 -- start start 2024-04-01T00:14:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb1b5c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb140b1930 0x7ffb1406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb1b5c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 0x7ffb1415d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb1b5c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb140cf570 con 0x7ffb140d24b0 2024-04-01T00:14:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb1b5c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1406d590 con 0x7ffb140b1930 2024-04-01T00:14:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb1b5c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ffb140b1f10 con 0x7ffb1406d120 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb18b64640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb140b1930 0x7ffb1406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb19365640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffb140d24b0 0x7ffb140cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:51928/0 (socket says 172.21.15.64:51928) 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb19365640 1 -- 172.21.15.64:0/346453252 learned_addr learned my addr 172.21.15.64:0/346453252 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb19b66640 1 --2- 172.21.15.64:0/346453252 >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 0x7ffb1415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb0bfff640 1 -- 172.21.15.64:0/346453252 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013691782 0 0) 0x7ffb140cf570 con 0x7ffb140d24b0 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb18b64640 1 -- 172.21.15.64:0/346453252 >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 msgr2=0x7ffb1415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb18b64640 1 --2- 172.21.15.64:0/346453252 >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 0x7ffb1415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb18b64640 1 -- 172.21.15.64:0/346453252 >> v1:172.21.15.100:6789/0 conn(0x7ffb140d24b0 legacy=0x7ffb140cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb18b64640 1 -- 172.21.15.64:0/346453252 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb140b21f0 con 0x7ffb140b1930 2024-04-01T00:14:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.608+0000 7ffb19b66640 1 --2- 172.21.15.64:0/346453252 >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 0x7ffb1415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.609+0000 7ffb18b64640 1 --2- 172.21.15.64:0/346453252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb140b1930 0x7ffb1406cbe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb14162ea0 tx=0x7ffb0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8c230e0a74fbde server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.609+0000 7ffb0bfff640 1 -- 172.21.15.64:0/346453252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04035020 con 0x7ffb140b1930 2024-04-01T00:14:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.609+0000 7ffb0bfff640 1 -- 172.21.15.64:0/346453252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb04031830 con 0x7ffb140b1930 2024-04-01T00:14:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.609+0000 7ffb0bfff640 1 -- 172.21.15.64:0/346453252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04031c00 con 0x7ffb140b1930 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/346453252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb140b1930 msgr2=0x7ffb1406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/346453252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb140b1930 0x7ffb1406cbe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb14162ea0 tx=0x7ffb0402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/346453252 shutdown_connections 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/346453252 >> v2:172.21.15.100:3300/0 conn(0x7ffb1406d120 0x7ffb1415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/346453252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb140b1930 0x7ffb1406cbe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/346453252 >> 172.21.15.64:0/346453252 conn(0x7ffb1409a980 msgr2=0x7ffb1409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/346453252 shutdown_connections 2024-04-01T00:14:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.610+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/346453252 wait complete. 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.611+0000 7ffb1b5c9640 1 Processor -- start 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.611+0000 7ffb1b5c9640 1 -- start start 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb140b1930 0x7ffb1415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb140d24b0 0x7ffb1415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb140b2a90 con 0x7ffb1406d120 2024-04-01T00:14:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb140b27b0 con 0x7ffb140d24b0 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb1b5c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffb140d2890 con 0x7ffb140b1930 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47200/0 (socket says 172.21.15.64:47200) 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 -- 172.21.15.64:0/1921928331 learned_addr learned my addr 172.21.15.64:0/1921928331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19b66640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb140d24b0 0x7ffb1415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb18b64640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb140b1930 0x7ffb1415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 -- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb140b1930 msgr2=0x7ffb1415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb140b1930 0x7ffb1415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 -- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb140d24b0 msgr2=0x7ffb1415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb140d24b0 0x7ffb1415cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 -- 172.21.15.64:0/1921928331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1415be40 con 0x7ffb1406d120 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.612+0000 7ffb19365640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c430 tx=0x7ffb0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.613+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0074e0 con 0x7ffb1406d120 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.613+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0047e0 con 0x7ffb1406d120 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.613+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1415d870 con 0x7ffb1406d120 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.613+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c004bd0 con 0x7ffb1406d120 2024-04-01T00:14:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.613+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141f78e0 con 0x7ffb1406d120 2024-04-01T00:14:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.614+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb140681a0 con 0x7ffb1406d120 2024-04-01T00:14:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.614+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ffb0c011070 con 0x7ffb1406d120 2024-04-01T00:14:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.614+0000 7ffb09ffb640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 0x7ffaf806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.615+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ffb0c08a300 con 0x7ffb1406d120 2024-04-01T00:14:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.615+0000 7ffb18b64640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 0x7ffaf806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.615+0000 7ffb18b64640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 0x7ffaf806de50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb04036000 tx=0x7ffb0403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.619+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ffb0c058f20 con 0x7ffb1406d120 2024-04-01T00:14:31.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.818+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffb140b1710 con 0x7ffb1406d120 2024-04-01T00:14:31.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.818+0000 7ffb09ffb640 1 -- 172.21.15.64:0/1921928331 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ffb0c058820 con 0x7ffb1406d120 2024-04-01T00:14:31.819 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-04-01T00:14:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 msgr2=0x7ffaf806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 0x7ffaf806de50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb04036000 tx=0x7ffb0403c040 comp rx=0 tx=0).stop 2024-04-01T00:14:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 msgr2=0x7ffb141f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c430 tx=0x7ffb0c00c830 comp rx=0 tx=0).stop 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 shutdown_connections 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffaf806b990 0x7ffaf806de50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb140d24b0 0x7ffb1415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb140b1930 0x7ffb1415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 --2- 172.21.15.64:0/1921928331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb1406d120 0x7ffb141f83c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 >> 172.21.15.64:0/1921928331 conn(0x7ffb1409a980 msgr2=0x7ffb1406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 shutdown_connections 2024-04-01T00:14:31.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:31.820+0000 7ffb1b5c9640 1 -- 172.21.15.64:0/1921928331 wait complete. 2024-04-01T00:14:31.832 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-01T00:14:31.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:31.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-01T00:14:31.860 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:31.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T00:14:31.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:31.861 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-01T00:14:31.915 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-01T00:14:31.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:32.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.017+0000 7f68c891e640 1 Processor -- start 2024-04-01T00:14:32.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.017+0000 7f68c891e640 1 -- start start 2024-04-01T00:14:32.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c891e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c415bb70 0x7f68c415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c891e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68c40c9cf0 0x7f68c40ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c891e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c415bf50 con 0x7f68c406d560 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c891e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f68c40afe60 con 0x7f68c40c9cf0 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c891e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68c415eb90 con 0x7f68c415bb70 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68c40c9cf0 0x7f68c40ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68c40c9cf0 0x7f68c40ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47206/0 (socket says 172.21.15.64:47206) 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c2d9d640 1 -- 172.21.15.64:0/3152734207 learned_addr learned my addr 172.21.15.64:0/3152734207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.018+0000 7f68c259c640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c415bb70 0x7f68c415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c2d9d640 1 -- 172.21.15.64:0/3152734207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c415bb70 msgr2=0x7f68c415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c2d9d640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c415bb70 0x7f68c415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c2d9d640 1 -- 172.21.15.64:0/3152734207 >> v1:172.21.15.100:6789/0 conn(0x7f68c406d560 legacy=0x7f68c4071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c2d9d640 1 -- 172.21.15.64:0/3152734207 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c415ee70 con 0x7f68c40c9cf0 2024-04-01T00:14:32.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c2d9d640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c40ca180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009870 tx=0x7f68b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209470f860b83467 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.019+0000 7f68c159a640 1 -- 172.21.15.64:0/3152734207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4034070 con 0x7f68c40c9cf0 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c159a640 1 -- 172.21.15.64:0/3152734207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b4030840 con 0x7f68c40c9cf0 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c159a640 1 -- 172.21.15.64:0/3152734207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4030c10 con 0x7f68c40c9cf0 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 -- 172.21.15.64:0/3152734207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 msgr2=0x7f68c40ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c40ca180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009870 tx=0x7f68b402b980 comp rx=0 tx=0).stop 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 -- 172.21.15.64:0/3152734207 shutdown_connections 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c40ca180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 --2- 172.21.15.64:0/3152734207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c415bb70 0x7f68c415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 -- 172.21.15.64:0/3152734207 >> 172.21.15.64:0/3152734207 conn(0x7f68c40d03e0 msgr2=0x7f68c40d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 -- 172.21.15.64:0/3152734207 shutdown_connections 2024-04-01T00:14:32.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.020+0000 7f68c891e640 1 -- 172.21.15.64:0/3152734207 wait complete. 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.021+0000 7f68c891e640 1 Processor -- start 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 -- start start 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c406d560 0x7f68c415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c415bb70 0x7f68c415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68c406d9c0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68c40c97d0 con 0x7f68c415bb70 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c891e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68c4071c50 con 0x7f68c406d560 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47218/0 (socket says 172.21.15.64:47218) 2024-04-01T00:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c1d9b640 1 -- 172.21.15.64:0/4169248776 learned_addr learned my addr 172.21.15.64:0/4169248776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c259c640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c406d560 0x7f68c415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.022+0000 7f68c2d9d640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c415bb70 0x7f68c415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 -- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c406d560 msgr2=0x7f68c415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c406d560 0x7f68c415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 -- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c415bb70 msgr2=0x7f68c415d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c415bb70 0x7f68c415d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 -- 172.21.15.64:0/4169248776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c40b4c30 con 0x7f68c40c9cf0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c1d9b640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c400 tx=0x7f68ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0052d0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac005490 con 0x7f68c40c9cf0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c40b4f10 con 0x7f68c40c9cf0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac007260 con 0x7f68c40c9cf0 2024-04-01T00:14:32.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.023+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c40b54d0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.024+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c40681a0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.024+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f68ac007400 con 0x7f68c40c9cf0 2024-04-01T00:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.025+0000 7f68baffd640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 0x7f68a406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.025+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f68ac089d80 con 0x7f68c40c9cf0 2024-04-01T00:14:32.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.025+0000 7f68c259c640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 0x7f68a406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.026+0000 7f68c259c640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 0x7f68a406de50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68bc011340 tx=0x7f68bc00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.029+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f68ac0582c0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.253+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68c41614b0 con 0x7f68c40c9cf0 2024-04-01T00:14:32.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.253+0000 7f68baffd640 1 -- 172.21.15.64:0/4169248776 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f68ac05c360 con 0x7f68c40c9cf0 2024-04-01T00:14:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:32.254 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 msgr2=0x7f68a406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 0x7f68a406de50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68bc011340 tx=0x7f68bc00b040 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 msgr2=0x7f68c415d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c400 tx=0x7f68ac00c8c0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 shutdown_connections 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f68a406b990 0x7f68a406de50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c415bb70 0x7f68c415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c40c9cf0 0x7f68c415d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 --2- 172.21.15.64:0/4169248776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68c406d560 0x7f68c415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 >> 172.21.15.64:0/4169248776 conn(0x7f68c40d03e0 msgr2=0x7f68c406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.255+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 shutdown_connections 2024-04-01T00:14:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.256+0000 7f68c891e640 1 -- 172.21.15.64:0/4169248776 wait complete. 2024-04-01T00:14:32.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.368+0000 7fcaf30d4640 1 Processor -- start 2024-04-01T00:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.368+0000 7fcaf30d4640 1 -- start start 2024-04-01T00:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec15bb70 0x7fcaec15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec06d560 0x7fcaec071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 0x7fcaec0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0afe60 con 0x7fcaec06d560 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec15bf50 con 0x7fcaec071a80 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf30d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec15eb90 con 0x7fcaec15bb70 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec06d560 0x7fcaec071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec06d560 0x7fcaec071540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47224/0 (socket says 172.21.15.64:47224) 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 -- 172.21.15.64:0/793914167 learned_addr learned my addr 172.21.15.64:0/793914167 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf1671640 1 --2- 172.21.15.64:0/793914167 >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 0x7fcaec0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf0e70640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec15bb70 0x7fcaec15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 -- 172.21.15.64:0/793914167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec15bb70 msgr2=0x7fcaec15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec15bb70 0x7fcaec15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 -- 172.21.15.64:0/793914167 >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 msgr2=0x7fcaec0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 --2- 172.21.15.64:0/793914167 >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 0x7fcaec0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaebfff640 1 -- 172.21.15.64:0/793914167 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec15ee70 con 0x7fcaec06d560 2024-04-01T00:14:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.369+0000 7fcaf1671640 1 --2- 172.21.15.64:0/793914167 >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 0x7fcaec0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.370+0000 7fcaebfff640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec06d560 0x7fcaec071540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0d13b0 tx=0x7fcad402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc640df528147e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.370+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/793914167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4035020 con 0x7fcaec06d560 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.370+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/793914167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4031870 con 0x7fcaec06d560 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.370+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/793914167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4031c40 con 0x7fcaec06d560 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 -- 172.21.15.64:0/793914167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec06d560 msgr2=0x7fcaec071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec06d560 0x7fcaec071540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0d13b0 tx=0x7fcad402b910 comp rx=0 tx=0).stop 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 -- 172.21.15.64:0/793914167 shutdown_connections 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/793914167 >> v2:172.21.15.100:3300/0 conn(0x7fcaec071a80 0x7fcaec0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec06d560 0x7fcaec071540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/793914167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec15bb70 0x7fcaec15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 -- 172.21.15.64:0/793914167 >> 172.21.15.64:0/793914167 conn(0x7fcaec0d03e0 msgr2=0x7fcaec0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 -- 172.21.15.64:0/793914167 shutdown_connections 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.371+0000 7fcaf30d4640 1 -- 172.21.15.64:0/793914167 wait complete. 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.372+0000 7fcaf30d4640 1 Processor -- start 2024-04-01T00:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.372+0000 7fcaf30d4640 1 -- start start 2024-04-01T00:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec06d560 0x7fcaec1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec15bb70 0x7fcaec14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06d9c0 con 0x7fcaec071a80 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0c97d0 con 0x7fcaec15bb70 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf30d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec15f430 con 0x7fcaec06d560 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47228/0 (socket says 172.21.15.64:47228) 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 -- 172.21.15.64:0/944948151 learned_addr learned my addr 172.21.15.64:0/944948151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf1671640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec15bb70 0x7fcaec14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaf0e70640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec06d560 0x7fcaec1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 -- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec06d560 msgr2=0x7fcaec1f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec06d560 0x7fcaec1f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 -- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec15bb70 msgr2=0x7fcaec14ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec15bb70 0x7fcaec14ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.373+0000 7fcaebfff640 1 -- 172.21.15.64:0/944948151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0b4280 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcaebfff640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcad402b950 tx=0x7fcad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4040070 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4031e10 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4035040 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec15e030 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.374+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec0b3fa0 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.375+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec0681a0 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.375+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fcad404b060 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.376+0000 7fcae97fa640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 0x7fcacc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.376+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcad40ac490 con 0x7fcaec071a80 2024-04-01T00:14:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.376+0000 7fcaf0e70640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 0x7fcacc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.376+0000 7fcaf0e70640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 0x7fcacc06de50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011340 tx=0x7fcae400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.380+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fcad407a950 con 0x7fcaec071a80 2024-04-01T00:14:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.565+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaec15d850 con 0x7fcaec071a80 2024-04-01T00:14:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.566+0000 7fcae97fa640 1 -- 172.21.15.64:0/944948151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16830 (secure 0 0 0) 0x7fcad407e9f0 con 0x7fcaec071a80 2024-04-01T00:14:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:28.115298+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 msgr2=0x7fcacc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 0x7fcacc06de50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011340 tx=0x7fcae400b040 comp rx=0 tx=0).stop 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 msgr2=0x7fcaec15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcad402b950 tx=0x7fcad40046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 shutdown_connections 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fcacc06b990 0x7fcacc06de50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec15bb70 0x7fcaec14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec071a80 0x7fcaec15daf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 --2- 172.21.15.64:0/944948151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcaec06d560 0x7fcaec1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 >> 172.21.15.64:0/944948151 conn(0x7fcaec0d03e0 msgr2=0x7fcaec0cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 shutdown_connections 2024-04-01T00:14:32.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.568+0000 7fcaf30d4640 1 -- 172.21.15.64:0/944948151 wait complete. 2024-04-01T00:14:32.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:13:44.828390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:13:45.342586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:32.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.684+0000 7f87e7c9c640 1 Processor -- start 2024-04-01T00:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.684+0000 7f87e7c9c640 1 -- start start 2024-04-01T00:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e7c9c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87e00b0ba0 0x7f87e00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e7c9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e7c9c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e00cc730 con 0x7f87e00b1140 2024-04-01T00:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e7c9c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f87e00b0f80 con 0x7f87e00b0ba0 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e7c9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87e00cca10 con 0x7f87e00cfe50 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87e00b0ba0 0x7f87e00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87e00b0ba0 0x7f87e00cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47232/0 (socket says 172.21.15.64:47232) 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 -- 172.21.15.64:0/3153326886 learned_addr learned my addr 172.21.15.64:0/3153326886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e6239640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 -- 172.21.15.64:0/3153326886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 msgr2=0x7f87e00b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e00b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 -- 172.21.15.64:0/3153326886 >> v1:172.21.15.100:6789/0 conn(0x7f87e00b1140 legacy=0x7f87e00cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e5a38640 1 -- 172.21.15.64:0/3153326886 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e00cccf0 con 0x7f87e00b0ba0 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.685+0000 7f87e4a36640 1 -- 172.21.15.64:0/3153326886 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404997439 0 0) 0x7f87e00cc730 con 0x7f87e00b1140 2024-04-01T00:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e5a38640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00cd7d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009870 tx=0x7f87dc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f5b7308dc6723f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e4a36640 1 -- 172.21.15.64:0/3153326886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc034070 con 0x7f87e00b0ba0 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e4a36640 1 -- 172.21.15.64:0/3153326886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc030840 con 0x7f87e00b0ba0 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e4a36640 1 -- 172.21.15.64:0/3153326886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc030c10 con 0x7f87e00b0ba0 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e7c9c640 1 -- 172.21.15.64:0/3153326886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 msgr2=0x7f87e00cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.686+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00cd7d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009870 tx=0x7f87dc02b980 comp rx=0 tx=0).stop 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 -- 172.21.15.64:0/3153326886 shutdown_connections 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e00b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/3153326886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00cd7d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 -- 172.21.15.64:0/3153326886 >> 172.21.15.64:0/3153326886 conn(0x7f87e0072810 msgr2=0x7f87e0072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 -- 172.21.15.64:0/3153326886 shutdown_connections 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.687+0000 7f87e7c9c640 1 -- 172.21.15.64:0/3153326886 wait complete. 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.688+0000 7f87e7c9c640 1 Processor -- start 2024-04-01T00:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.688+0000 7f87e7c9c640 1 -- start start 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 0x7f87e00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e0153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0162e10 con 0x7f87e00b0ba0 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87e00cd2b0 con 0x7f87e00b1140 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87e00caa30 con 0x7f87e00cfe50 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47240/0 (socket says 172.21.15.64:47240) 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 -- 172.21.15.64:0/2991559742 learned_addr learned my addr 172.21.15.64:0/2991559742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e6239640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e0153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5237640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 0x7f87e00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 -- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 msgr2=0x7f87e0153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e0153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 -- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 msgr2=0x7f87e00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 0x7f87e00c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 -- 172.21.15.64:0/2991559742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e00c4470 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e6239640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e0153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5237640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 0x7f87e00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e5a38640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0090b0 tx=0x7f87dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc034040 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc030640 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e00c4750 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc039d40 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.689+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e00c4d10 con 0x7f87e00b0ba0 2024-04-01T00:14:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.690+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e00cead0 con 0x7f87e00b0ba0 2024-04-01T00:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.691+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f87dc04b060 con 0x7f87e00b0ba0 2024-04-01T00:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.691+0000 7f87d67fc640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 0x7f87c406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.691+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f87dc0ac4d0 con 0x7f87e00b0ba0 2024-04-01T00:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.691+0000 7f87e5237640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 0x7f87c406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.692+0000 7f87e5237640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 0x7f87c406de50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87e0162080 tx=0x7f87d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.696+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f87e00cead0 con 0x7f87e00b0ba0 2024-04-01T00:14:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.876+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87e00681a0 con 0x7f87e00b0ba0 2024-04-01T00:14:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.876+0000 7f87d67fc640 1 -- 172.21.15.64:0/2991559742 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f87dc07b180 con 0x7f87e00b0ba0 2024-04-01T00:14:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 msgr2=0x7f87c406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 0x7f87c406de50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87e0162080 tx=0x7f87d0009040 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 msgr2=0x7f87e00c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0090b0 tx=0x7f87dc0046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 shutdown_connections 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f87c406b990 0x7f87c406de50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87e00cfe50 0x7f87e0153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87e00b1140 0x7f87e00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 --2- 172.21.15.64:0/2991559742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87e00b0ba0 0x7f87e00c3e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 >> 172.21.15.64:0/2991559742 conn(0x7f87e0072810 msgr2=0x7f87e006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 shutdown_connections 2024-04-01T00:14:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.878+0000 7f87e7c9c640 1 -- 172.21.15.64:0/2991559742 wait complete. 2024-04-01T00:14:32.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T00:14:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.994+0000 7f97b8af4640 1 Processor -- start 2024-04-01T00:14:33.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.994+0000 7f97b8af4640 1 -- start start 2024-04-01T00:14:33.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b8af4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b4144720 0x7f97b406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b8af4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97b40cab60 0x7f97b406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b8af4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97b40cb020 con 0x7f97b406eb30 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b8af4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f97b4144b00 con 0x7f97b40cab60 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b8af4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97b40cb300 con 0x7f97b4144720 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97b40cab60 0x7f97b406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97b40cab60 0x7f97b406e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47256/0 (socket says 172.21.15.64:47256) 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 -- 172.21.15.64:0/1916782225 learned_addr learned my addr 172.21.15.64:0/1916782225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b259c640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b4144720 0x7f97b406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 -- 172.21.15.64:0/1916782225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b4144720 msgr2=0x7f97b406ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b4144720 0x7f97b406ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 -- 172.21.15.64:0/1916782225 >> v1:172.21.15.100:6789/0 conn(0x7f97b406eb30 legacy=0x7f97b415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b159a640 1 -- 172.21.15.64:0/1916782225 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787486924 0 0) 0x7f97b40cb020 con 0x7f97b406eb30 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.995+0000 7f97b1d9b640 1 -- 172.21.15.64:0/1916782225 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b40cb5e0 con 0x7f97b40cab60 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b1d9b640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b40cab60 0x7f97b406e5f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97a800bb10 tx=0x7f97a802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2494a258a4df51 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b159a640 1 -- 172.21.15.64:0/1916782225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a803b070 con 0x7f97b40cab60 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b159a640 1 -- 172.21.15.64:0/1916782225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a8036440 con 0x7f97b40cab60 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b159a640 1 -- 172.21.15.64:0/1916782225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8036810 con 0x7f97b40cab60 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b8af4640 1 -- 172.21.15.64:0/1916782225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b40cab60 msgr2=0x7f97b406e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.996+0000 7f97b8af4640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b40cab60 0x7f97b406e5f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97a800bb10 tx=0x7f97a802e150 comp rx=0 tx=0).stop 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 -- 172.21.15.64:0/1916782225 shutdown_connections 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b40cab60 0x7f97b406e5f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 --2- 172.21.15.64:0/1916782225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b4144720 0x7f97b406ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 -- 172.21.15.64:0/1916782225 >> 172.21.15.64:0/1916782225 conn(0x7f97b40cfa90 msgr2=0x7f97b40cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 -- 172.21.15.64:0/1916782225 shutdown_connections 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.997+0000 7f97b8af4640 1 -- 172.21.15.64:0/1916782225 wait complete. 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.998+0000 7f97b8af4640 1 Processor -- start 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.998+0000 7f97b8af4640 1 -- start start 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b406eb30 0x7f97b415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b40cab60 0x7f97b415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97b40cd7f0 con 0x7f97b4144720 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97b40cbba0 con 0x7f97b40cab60 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97b40cd970 con 0x7f97b406eb30 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47272/0 (socket says 172.21.15.64:47272) 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 -- 172.21.15.64:0/879610213 learned_addr learned my addr 172.21.15.64:0/879610213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b259c640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b406eb30 0x7f97b415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b1d9b640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b40cab60 0x7f97b415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 -- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b406eb30 msgr2=0x7f97b415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b406eb30 0x7f97b415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 -- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b40cab60 msgr2=0x7f97b415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b40cab60 0x7f97b415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 -- 172.21.15.64:0/879610213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b415c8d0 con 0x7f97b4144720 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b2d9d640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c760 tx=0x7f97ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac003da0 con 0x7f97b4144720 2024-04-01T00:14:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac005070 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41427b0 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac005460 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:32.999+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b4142d70 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.000+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b4142fb0 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.001+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f97ac005600 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.001+0000 7f97a2ffd640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 0x7f979406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.001+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f97ac08a710 con 0x7f97b4144720 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.001+0000 7f97b259c640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 0x7f979406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.002+0000 7f97b259c640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 0x7f979406de50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009aa0 tx=0x7f97a4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.006+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f97b4142fb0 con 0x7f97b4144720 2024-04-01T00:14:33.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.188+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97b40681a0 con 0x7f97b4144720 2024-04-01T00:14:33.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.188+0000 7f97a2ffd640 1 -- 172.21.15.64:0/879610213 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f97ac058c50 con 0x7f97b4144720 2024-04-01T00:14:33.189 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T00:14:33.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 msgr2=0x7f979406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 0x7f979406de50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009aa0 tx=0x7f97a4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 msgr2=0x7f97b4141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c760 tx=0x7f97ac00cc20 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 shutdown_connections 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f979406b990 0x7f979406de50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4144720 0x7f97b4141d10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b40cab60 0x7f97b415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 --2- 172.21.15.64:0/879610213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b406eb30 0x7f97b415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 >> 172.21.15.64:0/879610213 conn(0x7f97b40cfa90 msgr2=0x7f97b4140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 shutdown_connections 2024-04-01T00:14:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.190+0000 7f97b8af4640 1 -- 172.21.15.64:0/879610213 wait complete. 2024-04-01T00:14:33.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T00:14:33.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.306+0000 7f15f7f3a640 1 Processor -- start 2024-04-01T00:14:33.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.306+0000 7f15f7f3a640 1 -- start start 2024-04-01T00:14:33.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.306+0000 7f15f7f3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f006ed00 0x7f15f015be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f7f3a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15f0070bf0 0x7f15f00b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f7f3a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f0071360 con 0x7f15f0158a70 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f7f3a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f15f0071640 con 0x7f15f0070bf0 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f7f3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15f0071920 con 0x7f15f006ed00 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15f0070bf0 0x7f15f00b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15f0070bf0 0x7f15f00b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47288/0 (socket says 172.21.15.64:47288) 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 -- 172.21.15.64:0/376959185 learned_addr learned my addr 172.21.15.64:0/376959185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f54d5640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f006ed00 0x7f15f015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 -- 172.21.15.64:0/376959185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f006ed00 msgr2=0x7f15f015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f006ed00 0x7f15f015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 -- 172.21.15.64:0/376959185 >> v1:172.21.15.100:6789/0 conn(0x7f15f0158a70 legacy=0x7f15f006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.307+0000 7f15f64d7640 1 -- 172.21.15.64:0/376959185 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f00b2c40 con 0x7f15f0070bf0 2024-04-01T00:14:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f64d7640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f0070bf0 0x7f15f00b3160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009e50 tx=0x7f15e802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ee9ce3674591a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f4cd4640 1 -- 172.21.15.64:0/376959185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e802cb00 con 0x7f15f0070bf0 2024-04-01T00:14:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f4cd4640 1 -- 172.21.15.64:0/376959185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e802cca0 con 0x7f15f0070bf0 2024-04-01T00:14:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f4cd4640 1 -- 172.21.15.64:0/376959185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8034610 con 0x7f15f0070bf0 2024-04-01T00:14:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 -- 172.21.15.64:0/376959185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f0070bf0 msgr2=0x7f15f00b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f0070bf0 0x7f15f00b3160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009e50 tx=0x7f15e802bf60 comp rx=0 tx=0).stop 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 -- 172.21.15.64:0/376959185 shutdown_connections 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f0070bf0 0x7f15f00b3160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/376959185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f006ed00 0x7f15f015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 -- 172.21.15.64:0/376959185 >> 172.21.15.64:0/376959185 conn(0x7f15f00b1730 msgr2=0x7f15f00b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 -- 172.21.15.64:0/376959185 shutdown_connections 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.308+0000 7f15f7f3a640 1 -- 172.21.15.64:0/376959185 wait complete. 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 Processor -- start 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 -- start start 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f0070bf0 0x7f15f0142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f0158a70 0x7f15f0152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00b3820 con 0x7f15f006ed00 2024-04-01T00:14:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00b36a0 con 0x7f15f0158a70 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f7f3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15f00b39a0 con 0x7f15f0070bf0 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f5cd6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f5cd6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47290/0 (socket says 172.21.15.64:47290) 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f5cd6640 1 -- 172.21.15.64:0/1938469506 learned_addr learned my addr 172.21.15.64:0/1938469506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f64d7640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f0158a70 0x7f15f0152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.310+0000 7f15f54d5640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f0070bf0 0x7f15f0142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 -- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f0070bf0 msgr2=0x7f15f0142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f0070bf0 0x7f15f0142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 -- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f0158a70 msgr2=0x7f15f0152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f0158a70 0x7f15f0152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 -- 172.21.15.64:0/1938469506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f0154ea0 con 0x7f15f006ed00 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f5cd6640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c3c0 tx=0x7f15ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0052d0 con 0x7f15f006ed00 2024-04-01T00:14:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec005490 con 0x7f15f006ed00 2024-04-01T00:14:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f0145920 con 0x7f15f006ed00 2024-04-01T00:14:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec007260 con 0x7f15f006ed00 2024-04-01T00:14:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.311+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f0142e00 con 0x7f15f006ed00 2024-04-01T00:14:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.312+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f00681a0 con 0x7f15f006ed00 2024-04-01T00:14:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.313+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f15ec007400 con 0x7f15f006ed00 2024-04-01T00:14:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.313+0000 7f15e27fc640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 0x7f15d406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.313+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f15ec089cf0 con 0x7f15f006ed00 2024-04-01T00:14:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.313+0000 7f15f54d5640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 0x7f15d406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.314+0000 7f15f54d5640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 0x7f15d406de50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15f00af410 tx=0x7f15e4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.317+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f15ec08e070 con 0x7f15f006ed00 2024-04-01T00:14:33.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.498+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f15f00af440 con 0x7f15f006ed00 2024-04-01T00:14:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.499+0000 7f15e27fc640 1 -- 172.21.15.64:0/1938469506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f15f0145920 con 0x7f15f006ed00 2024-04-01T00:14:33.500 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T00:14:33.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.500+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 msgr2=0x7f15d406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.500+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 0x7f15d406de50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15f00af410 tx=0x7f15e4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.500+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 msgr2=0x7f15f01453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.500+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c3c0 tx=0x7f15ec00c880 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 shutdown_connections 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f15d406b990 0x7f15d406de50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f0158a70 0x7f15f0152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f0070bf0 0x7f15f0142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 --2- 172.21.15.64:0/1938469506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15f006ed00 0x7f15f01453e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 >> 172.21.15.64:0/1938469506 conn(0x7f15f00b1730 msgr2=0x7f15f00b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 shutdown_connections 2024-04-01T00:14:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.501+0000 7f15f7f3a640 1 -- 172.21.15.64:0/1938469506 wait complete. 2024-04-01T00:14:33.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:33.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.613+0000 7f888941c640 1 Processor -- start 2024-04-01T00:14:33.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.613+0000 7f888941c640 1 -- start start 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f888941c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f888941c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88840cc340 0x7f88840d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f888941c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88840d0060 con 0x7f888406c390 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f888941c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f88840d0650 con 0x7f88840cc340 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f888941c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f888406d100 con 0x7f888415e1c0 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88827fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88840cc340 0x7f88840d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88837fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f888406c390 0x7f888406c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33314/0 (socket says 172.21.15.64:33314) 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88837fe640 1 -- 172.21.15.64:0/1983387329 learned_addr learned my addr 172.21.15.64:0/1983387329 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f8882ffd640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88827fc640 1 -- 172.21.15.64:0/1983387329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 msgr2=0x7f88840d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88827fc640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88827fc640 1 -- 172.21.15.64:0/1983387329 >> v1:172.21.15.64:6789/0 conn(0x7f888406c390 legacy=0x7f888406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f8881ffb640 1 -- 172.21.15.64:0/1983387329 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307959656 0 0) 0x7f8874001490 con 0x7f888406c390 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f88827fc640 1 -- 172.21.15.64:0/1983387329 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888406d3e0 con 0x7f88840cc340 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.614+0000 7f8882ffd640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.615+0000 7f88827fc640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88840cc340 0x7f88840d2150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f886c0097a0 tx=0x7f886c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351893b66cb1d004 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.615+0000 7f8881ffb640 1 -- 172.21.15.64:0/1983387329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c0315a0 con 0x7f88840cc340 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.615+0000 7f8881ffb640 1 -- 172.21.15.64:0/1983387329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c037070 con 0x7f88840cc340 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.615+0000 7f8881ffb640 1 -- 172.21.15.64:0/1983387329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c031960 con 0x7f88840cc340 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 -- 172.21.15.64:0/1983387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88840cc340 msgr2=0x7f88840d2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88840cc340 0x7f88840d2150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f886c0097a0 tx=0x7f886c02b840 comp rx=0 tx=0).stop 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 -- 172.21.15.64:0/1983387329 shutdown_connections 2024-04-01T00:14:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88840cc340 0x7f88840d2150 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 --2- 172.21.15.64:0/1983387329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 -- 172.21.15.64:0/1983387329 >> 172.21.15.64:0/1983387329 conn(0x7f888409a980 msgr2=0x7f888409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 -- 172.21.15.64:0/1983387329 shutdown_connections 2024-04-01T00:14:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.616+0000 7f888941c640 1 -- 172.21.15.64:0/1983387329 wait complete. 2024-04-01T00:14:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.617+0000 7f888941c640 1 Processor -- start 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.617+0000 7f888941c640 1 -- start start 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88840cc340 0x7f8884160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f88840b0b80 con 0x7f888406c390 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88840b0a00 con 0x7f88840cc340 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f888941c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88840b0d00 con 0x7f888415e1c0 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58128/0 (socket says 172.21.15.64:58128) 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 -- 172.21.15.64:0/1194743686 learned_addr learned my addr 172.21.15.64:0/1194743686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f88827fc640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88840cc340 0x7f8884160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f88837fe640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 -- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 msgr2=0x7f88840c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 -- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88840cc340 msgr2=0x7f8884160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88840cc340 0x7f8884160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 -- 172.21.15.64:0/1194743686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888415f570 con 0x7f888406c390 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.618+0000 7f8882ffd640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e960 tx=0x7f887c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.619+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f888406c390 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.619+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887c00ca00 con 0x7f888406c390 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.619+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888415f850 con 0x7f888406c390 2024-04-01T00:14:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.619+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00cdd0 con 0x7f888406c390 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.619+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888415fe10 con 0x7f888406c390 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.620+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88840681a0 con 0x7f888406c390 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.620+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f887c01a070 con 0x7f888406c390 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.620+0000 7f887b7fe640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 0x7f886406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.620+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f887c08aa60 con 0x7f888406c390 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.621+0000 7f88827fc640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 0x7f886406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.621+0000 7f88827fc640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 0x7f886406de80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f886c036000 tx=0x7f886c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.625+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f887c058fa0 con 0x7f888406c390 2024-04-01T00:14:33.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.841+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88840cd5d0 con 0x7f888406c390 2024-04-01T00:14:33.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.842+0000 7f887b7fe640 1 -- 172.21.15.64:0/1194743686 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f887c05d040 con 0x7f888406c390 2024-04-01T00:14:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:33.843 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 msgr2=0x7f886406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 0x7f886406de80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f886c036000 tx=0x7f886c03d040 comp rx=0 tx=0).stop 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 msgr2=0x7f888415aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e960 tx=0x7f887c00ee20 comp rx=0 tx=0).stop 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 shutdown_connections 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f886406b9c0 0x7f886406de80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f888415e1c0 0x7f88840c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88840cc340 0x7f8884160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 --2- 172.21.15.64:0/1194743686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f888406c390 0x7f888415aed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 >> 172.21.15.64:0/1194743686 conn(0x7f888409a980 msgr2=0x7f88840cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 shutdown_connections 2024-04-01T00:14:33.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.844+0000 7f888941c640 1 -- 172.21.15.64:0/1194743686 wait complete. 2024-04-01T00:14:33.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.956+0000 7f936e9f2640 1 Processor -- start 2024-04-01T00:14:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.956+0000 7f936e9f2640 1 -- start start 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936e9f2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93681619e0 0x7f93680d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936e9f2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 0x7f93681620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936e9f2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93680b3cb0 con 0x7f93680b3820 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936e9f2640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f93680d2ba0 con 0x7f93680cfae0 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936e9f2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f93680d2e80 con 0x7f93681619e0 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f936cf8f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 0x7f93681620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f93677fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f93680b3820 0x7f93680cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33326/0 (socket says 172.21.15.64:33326) 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f93677fe640 1 -- 172.21.15.64:0/2629985502 learned_addr learned my addr 172.21.15.64:0/2629985502 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.957+0000 7f9367fff640 1 --2- 172.21.15.64:0/2629985502 >> v2:172.21.15.100:3300/0 conn(0x7f93681619e0 0x7f93680d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9367fff640 1 -- 172.21.15.64:0/2629985502 >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 msgr2=0x7f93681620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9367fff640 1 --2- 172.21.15.64:0/2629985502 >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 0x7f93681620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9367fff640 1 -- 172.21.15.64:0/2629985502 >> v1:172.21.15.64:6789/0 conn(0x7f93680b3820 legacy=0x7f93680cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9367fff640 1 -- 172.21.15.64:0/2629985502 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680ceeb0 con 0x7f93681619e0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9366ffd640 1 -- 172.21.15.64:0/2629985502 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112852557 0 0) 0x7f93680b3cb0 con 0x7f93680b3820 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f936cf8f640 1 --2- 172.21.15.64:0/2629985502 >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 0x7f93681620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.958+0000 7f9367fff640 1 --2- 172.21.15.64:0/2629985502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680d4a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93680b6b40 tx=0x7f935802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aeddef801ddc626 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f9366ffd640 1 -- 172.21.15.64:0/2629985502 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580315a0 con 0x7f93681619e0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f9366ffd640 1 -- 172.21.15.64:0/2629985502 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9358037070 con 0x7f93681619e0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f9366ffd640 1 -- 172.21.15.64:0/2629985502 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358031960 con 0x7f93681619e0 2024-04-01T00:14:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 -- 172.21.15.64:0/2629985502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 msgr2=0x7f93680d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2629985502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680d4a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93680b6b40 tx=0x7f935802b840 comp rx=0 tx=0).stop 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 -- 172.21.15.64:0/2629985502 shutdown_connections 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2629985502 >> v2:172.21.15.64:3300/0 conn(0x7f93680cfae0 0x7f93681620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2629985502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680d4a60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 -- 172.21.15.64:0/2629985502 >> 172.21.15.64:0/2629985502 conn(0x7f936809d290 msgr2=0x7f93680d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 -- 172.21.15.64:0/2629985502 shutdown_connections 2024-04-01T00:14:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.959+0000 7f936e9f2640 1 -- 172.21.15.64:0/2629985502 wait complete. 2024-04-01T00:14:33.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.960+0000 7f936e9f2640 1 Processor -- start 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.960+0000 7f936e9f2640 1 -- start start 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680b3820 0x7f93681f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93680b31b0 con 0x7f93680cfae0 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93680d2890 con 0x7f93681619e0 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936e9f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f93680ceb10 con 0x7f93680b3820 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f93677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f93677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58158/0 (socket says 172.21.15.64:58158) 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f93677fe640 1 -- 172.21.15.64:0/2733524145 learned_addr learned my addr 172.21.15.64:0/2733524145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f936cf8f640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.961+0000 7f9367fff640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680b3820 0x7f93681f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 -- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680b3820 msgr2=0x7f93681f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680b3820 0x7f93681f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 -- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 msgr2=0x7f93680c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 -- 172.21.15.64:0/2733524145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680b7590 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f936cf8f640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f93677fe640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c4a0 tx=0x7f935c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c0074e0 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c0047e0 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93680b7870 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c004c50 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.962+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93680b7e30 con 0x7f93680cfae0 2024-04-01T00:14:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.963+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93680681a0 con 0x7f93680cfae0 2024-04-01T00:14:33.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.963+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f935c011070 con 0x7f93680cfae0 2024-04-01T00:14:33.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.964+0000 7f9364ff9640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 0x7f935406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:33.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.964+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f935c08a4b0 con 0x7f93680cfae0 2024-04-01T00:14:33.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.964+0000 7f9367fff640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 0x7f935406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:33.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.964+0000 7f9367fff640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 0x7f935406de50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9358036000 tx=0x7f9358008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:33.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:33.968+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f935c058fc0 con 0x7f93680cfae0 2024-04-01T00:14:34.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.154+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f936809d730 con 0x7f93680cfae0 2024-04-01T00:14:34.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.155+0000 7f9364ff9640 1 -- 172.21.15.64:0/2733524145 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16830 (secure 0 0 0) 0x7f935c0588c0 con 0x7f93680cfae0 2024-04-01T00:14:34.156 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:34.156 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:28.115298+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 msgr2=0x7f935406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 0x7f935406de50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9358036000 tx=0x7f9358008030 comp rx=0 tx=0).stop 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 msgr2=0x7f93680b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c4a0 tx=0x7f935c00c8a0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 shutdown_connections 2024-04-01T00:14:34.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f935406b990 0x7f935406de50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93681619e0 0x7f93680c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93680cfae0 0x7f93680b6bd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 --2- 172.21.15.64:0/2733524145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93680b3820 0x7f93681f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 >> 172.21.15.64:0/2733524145 conn(0x7f936809d290 msgr2=0x7f936806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 shutdown_connections 2024-04-01T00:14:34.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.157+0000 7f936e9f2640 1 -- 172.21.15.64:0/2733524145 wait complete. 2024-04-01T00:14:34.169 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-01T00:14:34.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.269+0000 7f4922fdd640 1 Processor -- start 2024-04-01T00:14:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.269+0000 7f4922fdd640 1 -- start start 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4922fdd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 0x7f491c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4922fdd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4922fdd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f491c15bf50 con 0x7f491c15bb70 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4922fdd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f491c0afe60 con 0x7f491c15e5b0 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4922fdd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f491c15eb90 con 0x7f491c0c9cf0 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f492157a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f491bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 0x7f491c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f492157a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:46380/0 (socket says 172.21.15.64:46380) 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f492157a640 1 -- 172.21.15.64:0/3569465256 learned_addr learned my addr 172.21.15.64:0/3569465256 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.270+0000 7f4920d79640 1 --1- 172.21.15.64:0/3569465256 >> v1:172.21.15.100:6789/0 conn(0x7f491c15bb70 0x7f491c071540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:49118/0 (socket says 172.21.15.64:49118) 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f492157a640 1 -- 172.21.15.64:0/3569465256 >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 msgr2=0x7f491c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f492157a640 1 --2- 172.21.15.64:0/3569465256 >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 0x7f491c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f492157a640 1 -- 172.21.15.64:0/3569465256 >> v1:172.21.15.100:6789/0 conn(0x7f491c15bb70 legacy=0x7f491c071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f492157a640 1 -- 172.21.15.64:0/3569465256 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c15ee70 con 0x7f491c0c9cf0 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f491b7fe640 1 -- 172.21.15.64:0/3569465256 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148888847 0 0) 0x7f491c15bf50 con 0x7f491c15bb70 2024-04-01T00:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.271+0000 7f491bfff640 1 --2- 172.21.15.64:0/3569465256 >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 0x7f491c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f492157a640 1 --2- 172.21.15.64:0/3569465256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4904009870 tx=0x7f490402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82a0e08c15973b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f491b7fe640 1 -- 172.21.15.64:0/3569465256 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904034070 con 0x7f491c0c9cf0 2024-04-01T00:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f491b7fe640 1 -- 172.21.15.64:0/3569465256 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4904030840 con 0x7f491c0c9cf0 2024-04-01T00:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f491b7fe640 1 -- 172.21.15.64:0/3569465256 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904030c10 con 0x7f491c0c9cf0 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 -- 172.21.15.64:0/3569465256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 msgr2=0x7f491c0ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 --2- 172.21.15.64:0/3569465256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4904009870 tx=0x7f490402b980 comp rx=0 tx=0).stop 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 -- 172.21.15.64:0/3569465256 shutdown_connections 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 --2- 172.21.15.64:0/3569465256 >> v2:172.21.15.100:3300/0 conn(0x7f491c15e5b0 0x7f491c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 --2- 172.21.15.64:0/3569465256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c0c9cf0 0x7f491c0ca180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 -- 172.21.15.64:0/3569465256 >> 172.21.15.64:0/3569465256 conn(0x7f491c0d03e0 msgr2=0x7f491c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 -- 172.21.15.64:0/3569465256 shutdown_connections 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.272+0000 7f4922fdd640 1 -- 172.21.15.64:0/3569465256 wait complete. 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.273+0000 7f4922fdd640 1 Processor -- start 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.273+0000 7f4922fdd640 1 -- start start 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15bb70 0x7f491c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15e5b0 0x7f491c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f491c071c50 con 0x7f491c0c9cf0 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0c97d0 con 0x7f491c15bb70 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4922fdd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f491c15f430 con 0x7f491c15e5b0 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4920d79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4920d79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58164/0 (socket says 172.21.15.64:58164) 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.274+0000 7f4920d79640 1 -- 172.21.15.64:0/1099937295 learned_addr learned my addr 172.21.15.64:0/1099937295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f491bfff640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15bb70 0x7f491c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f492157a640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15e5b0 0x7f491c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 -- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15e5b0 msgr2=0x7f491c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15e5b0 0x7f491c15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 -- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15bb70 msgr2=0x7f491c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15bb70 0x7f491c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 -- 172.21.15.64:0/1099937295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c15ccb0 con 0x7f491c0c9cf0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4920d79640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f491400c4d0 tx=0x7f491400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914018070 con 0x7f491c0c9cf0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4914007500 con 0x7f491c0c9cf0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c15cf90 con 0x7f491c0c9cf0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914004a20 con 0x7f491c0c9cf0 2024-04-01T00:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.275+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c0b36b0 con 0x7f491c0c9cf0 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.276+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c0681a0 con 0x7f491c0c9cf0 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.277+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f491401b070 con 0x7f491c0c9cf0 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.277+0000 7f49197fa640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 0x7f48fc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.277+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4914089b30 con 0x7f491c0c9cf0 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.277+0000 7f491bfff640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 0x7f48fc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.278+0000 7f491bfff640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 0x7f48fc06de50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f491c15e210 tx=0x7f490c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.281+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4914058070 con 0x7f491c0c9cf0 2024-04-01T00:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.503+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f491c0ce150 con 0x7f491c0c9cf0 2024-04-01T00:14:34.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.504+0000 7f49197fa640 1 -- 172.21.15.64:0/1099937295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f491405c110 con 0x7f491c0c9cf0 2024-04-01T00:14:34.505 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:34.506 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:34.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 msgr2=0x7f48fc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 0x7f48fc06de50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f491c15e210 tx=0x7f490c00b040 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 msgr2=0x7f491c149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f491400c4d0 tx=0x7f491400c990 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 shutdown_connections 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f48fc06b990 0x7f48fc06de50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15e5b0 0x7f491c15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15bb70 0x7f491c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 --2- 172.21.15.64:0/1099937295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c0c9cf0 0x7f491c149ea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.506+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 >> 172.21.15.64:0/1099937295 conn(0x7f491c0d03e0 msgr2=0x7f491c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.507+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 shutdown_connections 2024-04-01T00:14:34.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.507+0000 7f4922fdd640 1 -- 172.21.15.64:0/1099937295 wait complete. 2024-04-01T00:14:34.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:34.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.619+0000 7f4b14609640 1 Processor -- start 2024-04-01T00:14:34.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.619+0000 7f4b14609640 1 -- start start 2024-04-01T00:14:34.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b14609640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b0c15f610 0x7f4b0c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b14609640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b0c0cff40 0x7f4b0c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b14609640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c071460 con 0x7f4b0c151fe0 2024-04-01T00:14:34.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b14609640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c0d08e0 con 0x7f4b0c15f610 2024-04-01T00:14:34.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b14609640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c15f180 con 0x7f4b0c0cff40 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b123a5640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4b0c151fe0 0x7f4b0c070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33334/0 (socket says 172.21.15.64:33334) 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b11ba4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b0c15f610 0x7f4b0c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b123a5640 1 -- 172.21.15.64:0/26994578 learned_addr learned my addr 172.21.15.64:0/26994578 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b12ba6640 1 --2- 172.21.15.64:0/26994578 >> v2:172.21.15.100:3300/0 conn(0x7f4b0c0cff40 0x7f4b0c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b11ba4640 1 -- 172.21.15.64:0/26994578 >> v2:172.21.15.100:3300/0 conn(0x7f4b0c0cff40 msgr2=0x7f4b0c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b11ba4640 1 --2- 172.21.15.64:0/26994578 >> v2:172.21.15.100:3300/0 conn(0x7f4b0c0cff40 0x7f4b0c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b11ba4640 1 -- 172.21.15.64:0/26994578 >> v1:172.21.15.64:6789/0 conn(0x7f4b0c151fe0 legacy=0x7f4b0c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b11ba4640 1 -- 172.21.15.64:0/26994578 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c071740 con 0x7f4b0c15f610 2024-04-01T00:14:34.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.620+0000 7f4b113a3640 1 -- 172.21.15.64:0/26994578 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2192017835 0 0) 0x7f4b0c071460 con 0x7f4b0c151fe0 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.621+0000 7f4b11ba4640 1 --2- 172.21.15.64:0/26994578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c15beb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c06f180 tx=0x7f4afc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635e96e0a08e2c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.621+0000 7f4b113a3640 1 -- 172.21.15.64:0/26994578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc035020 con 0x7f4b0c15f610 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.621+0000 7f4b113a3640 1 -- 172.21.15.64:0/26994578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4afc031830 con 0x7f4b0c15f610 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.621+0000 7f4b113a3640 1 -- 172.21.15.64:0/26994578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc031c00 con 0x7f4b0c15f610 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 -- 172.21.15.64:0/26994578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 msgr2=0x7f4b0c15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 --2- 172.21.15.64:0/26994578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c15beb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c06f180 tx=0x7f4afc02b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 -- 172.21.15.64:0/26994578 shutdown_connections 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 --2- 172.21.15.64:0/26994578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c15beb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 --2- 172.21.15.64:0/26994578 >> v2:172.21.15.100:3300/0 conn(0x7f4b0c0cff40 0x7f4b0c0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 -- 172.21.15.64:0/26994578 >> 172.21.15.64:0/26994578 conn(0x7f4b0c06b4e0 msgr2=0x7f4b0c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 -- 172.21.15.64:0/26994578 shutdown_connections 2024-04-01T00:14:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.622+0000 7f4b14609640 1 -- 172.21.15.64:0/26994578 wait complete. 2024-04-01T00:14:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.623+0000 7f4b14609640 1 Processor -- start 2024-04-01T00:14:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.623+0000 7f4b14609640 1 -- start start 2024-04-01T00:14:34.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b0c0cff40 0x7f4b0c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0c151fe0 0x7f4b0c14b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c164af0 con 0x7f4b0c15f610 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c15ee10 con 0x7f4b0c151fe0 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b14609640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c164c70 con 0x7f4b0c0cff40 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58188/0 (socket says 172.21.15.64:58188) 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 -- 172.21.15.64:0/4188241659 learned_addr learned my addr 172.21.15.64:0/4188241659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b11ba4640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0c151fe0 0x7f4b0c14b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b123a5640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b0c0cff40 0x7f4b0c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 -- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b0c0cff40 msgr2=0x7f4b0c0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b0c0cff40 0x7f4b0c0ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 -- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0c151fe0 msgr2=0x7f4b0c14b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0c151fe0 0x7f4b0c14b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.624+0000 7f4b12ba6640 1 -- 172.21.15.64:0/4188241659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0adb00 con 0x7f4b0c15f610 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b12ba6640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400efc0 tx=0x7f4b0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04018020 con 0x7f4b0c15f610 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b04014480 con 0x7f4b0c15f610 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04014870 con 0x7f4b0c15f610 2024-04-01T00:14:34.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c14cf10 con 0x7f4b0c15f610 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.625+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c0ae420 con 0x7f4b0c15f610 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.626+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c15b980 con 0x7f4b0c15f610 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.626+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f4b04005d10 con 0x7f4b0c15f610 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.626+0000 7f4b02ffd640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 0x7f4af006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.627+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4b0408b260 con 0x7f4b0c15f610 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.627+0000 7f4b123a5640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 0x7f4af006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.627+0000 7f4b123a5640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 0x7f4af006de50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000f60 tx=0x7f4b08007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.631+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4b0c15b980 con 0x7f4b0c15f610 2024-04-01T00:14:34.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.850+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b0c0681a0 con 0x7f4b0c15f610 2024-04-01T00:14:34.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.851+0000 7f4b02ffd640 1 -- 172.21.15.64:0/4188241659 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f4b04022070 con 0x7f4b0c15f610 2024-04-01T00:14:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:34.852 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/4238865115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4238865115},{"type":"v1","addr":"172.21.15.64:6834","nonce":4238865115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1029873997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1029873997},{"type":"v1","addr":"172.21.15.64:6835","nonce":1029873997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/468132573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":468132573},{"type":"v1","addr":"172.21.15.100:6837","nonce":468132573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:13:46.387702+0000","modified":"2024-04-01T00:13:48.671456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2705635055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2705635055},{"type":"v1","addr":"172.21.15.100:6835","nonce":2705635055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-01T00:14:34.852 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 msgr2=0x7f4af006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 0x7f4af006de50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000f60 tx=0x7f4b08007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 msgr2=0x7f4b0c163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400efc0 tx=0x7f4b0400c490 comp rx=0 tx=0).stop 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 shutdown_connections 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4af006b990 0x7f4af006de50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b0c15f610 0x7f4b0c163eb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0c151fe0 0x7f4b0c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 --2- 172.21.15.64:0/4188241659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b0c0cff40 0x7f4b0c0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 >> 172.21.15.64:0/4188241659 conn(0x7f4b0c06b4e0 msgr2=0x7f4b0c0ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 shutdown_connections 2024-04-01T00:14:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.853+0000 7f4b14609640 1 -- 172.21.15.64:0/4188241659 wait complete. 2024-04-01T00:14:34.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.966+0000 7fb9e448f640 1 Processor -- start 2024-04-01T00:14:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.966+0000 7fb9e448f640 1 -- start start 2024-04-01T00:14:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e448f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9dc15e3f0 0x7fb9dc0cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e448f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0d2780 con 0x7fb9dc06c3e0 2024-04-01T00:14:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e448f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0ce790 con 0x7fb9dc0d1920 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e448f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc0cf3a0 con 0x7fb9dc15e3f0 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e1a2a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9dc15e3f0 0x7fb9dc0cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e2a2c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb9dc06c3e0 0x7fb9dc0d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33342/0 (socket says 172.21.15.64:33342) 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e2a2c640 1 -- 172.21.15.64:0/3510710669 learned_addr learned my addr 172.21.15.64:0/3510710669 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e1a2a640 1 -- 172.21.15.64:0/3510710669 >> v1:172.21.15.100:6789/0 conn(0x7fb9dc0d1920 legacy=0x7fb9dc0d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e1a2a640 1 -- 172.21.15.64:0/3510710669 >> v1:172.21.15.64:6789/0 conn(0x7fb9dc06c3e0 legacy=0x7fb9dc0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e1a2a640 1 -- 172.21.15.64:0/3510710669 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc0cea70 con 0x7fb9dc15e3f0 2024-04-01T00:14:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.967+0000 7fb9e1229640 1 -- 172.21.15.64:0/3510710669 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955060408 0 0) 0x7fb9dc0d2780 con 0x7fb9dc06c3e0 2024-04-01T00:14:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.968+0000 7fb9e1a2a640 1 --2- 172.21.15.64:0/3510710669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc15e3f0 0x7fb9dc0cff00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097a0 tx=0x7fb9d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff12db99e1d7204b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.968+0000 7fb9e1229640 1 -- 172.21.15.64:0/3510710669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8034070 con 0x7fb9dc15e3f0 2024-04-01T00:14:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.968+0000 7fb9e1229640 1 -- 172.21.15.64:0/3510710669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d8030610 con 0x7fb9dc15e3f0 2024-04-01T00:14:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.968+0000 7fb9e1229640 1 -- 172.21.15.64:0/3510710669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80309e0 con 0x7fb9dc15e3f0 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 -- 172.21.15.64:0/3510710669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc15e3f0 msgr2=0x7fb9dc0cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 --2- 172.21.15.64:0/3510710669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc15e3f0 0x7fb9dc0cff00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097a0 tx=0x7fb9d802b960 comp rx=0 tx=0).stop 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 -- 172.21.15.64:0/3510710669 shutdown_connections 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 --2- 172.21.15.64:0/3510710669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc15e3f0 0x7fb9dc0cff00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 -- 172.21.15.64:0/3510710669 >> 172.21.15.64:0/3510710669 conn(0x7fb9dc09a980 msgr2=0x7fb9dc09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 -- 172.21.15.64:0/3510710669 shutdown_connections 2024-04-01T00:14:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.969+0000 7fb9e448f640 1 -- 172.21.15.64:0/3510710669 wait complete. 2024-04-01T00:14:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.970+0000 7fb9e448f640 1 Processor -- start 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.970+0000 7fb9e448f640 1 -- start start 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9dc06c3e0 0x7fb9dc0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9dc0d1920 0x7fb9dc09ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0b0cd0 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0cf030 con 0x7fb9dc0d1920 2024-04-01T00:14:34.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e448f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc0b0e50 con 0x7fb9dc06c3e0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58206/0 (socket says 172.21.15.64:58206) 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 -- 172.21.15.64:0/1395120734 learned_addr learned my addr 172.21.15.64:0/1395120734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e1a2a640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9dc0d1920 0x7fb9dc09ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e222b640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9dc06c3e0 0x7fb9dc0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 -- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9dc06c3e0 msgr2=0x7fb9dc0af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9dc06c3e0 0x7fb9dc0af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 -- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9dc0d1920 msgr2=0x7fb9dc09ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9dc0d1920 0x7fb9dc09ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 -- 172.21.15.64:0/1395120734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1f7640 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9e2a2c640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7d0 tx=0x7fb9d400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4018070 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.971+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d400a640 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.972+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1f7920 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.972+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400aa70 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.972+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1f7ee0 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.973+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc0681a0 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.973+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb9d401b070 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.973+0000 7fb9ceffd640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 0x7fb9c006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:34.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.973+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb9d4089e50 con 0x7fb9dc09bfc0 2024-04-01T00:14:34.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.974+0000 7fb9e222b640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 0x7fb9c006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:34.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.974+0000 7fb9e222b640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 0x7fb9c006de50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0000f60 tx=0x7fb9d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:34.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:34.978+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb9d4058390 con 0x7fb9dc09bfc0 2024-04-01T00:14:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.163+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9dc0702e0 con 0x7fb9dc09bfc0 2024-04-01T00:14:35.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.164+0000 7fb9ceffd640 1 -- 172.21.15.64:0/1395120734 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16830 (secure 0 0 0) 0x7fb9d405c430 con 0x7fb9dc09bfc0 2024-04-01T00:14:35.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:35.166 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:28.115298+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:13:44.828390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-01T00:13:45.342586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 msgr2=0x7fb9c006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 0x7fb9c006de50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0000f60 tx=0x7fb9d0009040 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 msgr2=0x7fb9dc09c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7d0 tx=0x7fb9d400cc90 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 shutdown_connections 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb9c006b990 0x7fb9c006de50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc09bfc0 0x7fb9dc09c870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9dc0d1920 0x7fb9dc09ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 --2- 172.21.15.64:0/1395120734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9dc06c3e0 0x7fb9dc0af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.166+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 >> 172.21.15.64:0/1395120734 conn(0x7fb9dc09a980 msgr2=0x7fb9dc06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.167+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 shutdown_connections 2024-04-01T00:14:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.167+0000 7fb9e448f640 1 -- 172.21.15.64:0/1395120734 wait complete. 2024-04-01T00:14:35.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-01T00:14:35.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.280+0000 7f7ec118a640 1 Processor -- start 2024-04-01T00:14:35.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.280+0000 7f7ec118a640 1 -- start start 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc144720 0x7f7ebc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7ebc06eb30 0x7f7ebc15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc144b00 con 0x7f7ebc0cab60 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc0cb020 con 0x7f7ebc06eb30 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ec118a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cb300 con 0x7f7ebc144720 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7eba59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7eba59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58220/0 (socket says 172.21.15.64:58220) 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7eba59c640 1 -- 172.21.15.64:0/1210412881 learned_addr learned my addr 172.21.15.64:0/1210412881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ebb59e640 1 --2- 172.21.15.64:0/1210412881 >> v2:172.21.15.100:3300/0 conn(0x7f7ebc06eb30 0x7f7ebc15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.281+0000 7f7ebad9d640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc144720 0x7f7ebc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 -- 172.21.15.64:0/1210412881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc144720 msgr2=0x7f7ebc06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:35.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc144720 0x7f7ebc06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 -- 172.21.15.64:0/1210412881 >> v2:172.21.15.100:3300/0 conn(0x7f7ebc06eb30 msgr2=0x7f7ebc15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:35.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 --2- 172.21.15.64:0/1210412881 >> v2:172.21.15.100:3300/0 conn(0x7f7ebc06eb30 0x7f7ebc15e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 -- 172.21.15.64:0/1210412881 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc0cb5e0 con 0x7f7ebc0cab60 2024-04-01T00:14:35.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.282+0000 7f7eba59c640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0cddd0 tx=0x7f7ea402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfa207268ebcf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7eb9d9b640 1 -- 172.21.15.64:0/1210412881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4035020 con 0x7f7ebc0cab60 2024-04-01T00:14:35.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7eb9d9b640 1 -- 172.21.15.64:0/1210412881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4031870 con 0x7f7ebc0cab60 2024-04-01T00:14:35.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7eb9d9b640 1 -- 172.21.15.64:0/1210412881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4031c40 con 0x7f7ebc0cab60 2024-04-01T00:14:35.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 -- 172.21.15.64:0/1210412881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc0cab60 msgr2=0x7f7ebc06e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0cddd0 tx=0x7f7ea402b910 comp rx=0 tx=0).stop 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 -- 172.21.15.64:0/1210412881 shutdown_connections 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 --2- 172.21.15.64:0/1210412881 >> v2:172.21.15.100:3300/0 conn(0x7f7ebc06eb30 0x7f7ebc15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc06e5f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 --2- 172.21.15.64:0/1210412881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc144720 0x7f7ebc06ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 -- 172.21.15.64:0/1210412881 >> 172.21.15.64:0/1210412881 conn(0x7f7ebc0cfa90 msgr2=0x7f7ebc0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 -- 172.21.15.64:0/1210412881 shutdown_connections 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.283+0000 7f7ec118a640 1 -- 172.21.15.64:0/1210412881 wait complete. 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 Processor -- start 2024-04-01T00:14:35.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 -- start start 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc06eb30 0x7f7ebc0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cd7f0 con 0x7f7ebc144720 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cbba0 con 0x7f7ebc0cab60 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.285+0000 7f7ec118a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cd970 con 0x7f7ebc06eb30 2024-04-01T00:14:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58234/0 (socket says 172.21.15.64:58234) 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 -- 172.21.15.64:0/866276615 learned_addr learned my addr 172.21.15.64:0/866276615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7eba59c640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebad9d640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc06eb30 0x7f7ebc0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 -- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc06eb30 msgr2=0x7f7ebc0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc06eb30 0x7f7ebc0b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 -- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ebc0cab60 msgr2=0x7f7ebc15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 -- 172.21.15.64:0/866276615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc0c72c0 con 0x7f7ebc144720 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ebb59e640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010ef0 tx=0x7f7eac00e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac013650 con 0x7f7ebc144720 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac0137f0 con 0x7f7ebc144720 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc0c75a0 con 0x7f7ebc144720 2024-04-01T00:14:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac013be0 con 0x7f7ebc144720 2024-04-01T00:14:35.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.286+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc142b00 con 0x7f7ebc144720 2024-04-01T00:14:35.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.287+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc0681a0 con 0x7f7ebc144720 2024-04-01T00:14:35.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.288+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7eac01c400 con 0x7f7ebc144720 2024-04-01T00:14:35.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.288+0000 7f7eb37fe640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 0x7f7e9c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.288+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7eac08d5a0 con 0x7f7ebc144720 2024-04-01T00:14:35.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.288+0000 7f7ebad9d640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 0x7f7e9c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.289+0000 7f7ebad9d640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 0x7f7e9c06de50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009aa0 tx=0x7f7eb4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.292+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f7eac018030 con 0x7f7ebc144720 2024-04-01T00:14:35.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.511+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7ebc0cf230 con 0x7f7ebc144720 2024-04-01T00:14:35.512 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-04-01T00:14:35.512+0000 7f75af780640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-01T00:14:35.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.513+0000 7f7eb37fe640 1 -- 172.21.15.64:0/866276615 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f7eac05bb70 con 0x7f7ebc144720 2024-04-01T00:14:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.515+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 msgr2=0x7f7e9c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.515+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 0x7f7e9c06de50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009aa0 tx=0x7f7eb4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.515+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 msgr2=0x7f7ebc136f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.515+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010ef0 tx=0x7f7eac00e620 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 shutdown_connections 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7e9c06b990 0x7f7e9c06de50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ebc144720 0x7f7ebc136f20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ebc0cab60 0x7f7ebc15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 --2- 172.21.15.64:0/866276615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ebc06eb30 0x7f7ebc0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 >> 172.21.15.64:0/866276615 conn(0x7f7ebc0cfa90 msgr2=0x7f7ebc0c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 shutdown_connections 2024-04-01T00:14:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.516+0000 7f7ec118a640 1 -- 172.21.15.64:0/866276615 wait complete. 2024-04-01T00:14:35.518 INFO:tasks.ceph.mds.b.smithi100.stderr: -3224> 2024-04-01T00:13:47.319+0000 7fe784f9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-01T00:14:35.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-01T00:14:35.546 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-01T00:14:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.631+0000 7fb30e1c2640 1 Processor -- start 2024-04-01T00:14:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.631+0000 7fb30e1c2640 1 -- start start 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb30e1c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb3080b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb30e1c2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb308161e10 con 0x7fb3080d0900 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb30e1c2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3080d2880 con 0x7fb308161a30 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb30e1c2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb3080cef20 con 0x7fb3080d4a40 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb307fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb3080d0900 0x7fb3080cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33356/0 (socket says 172.21.15.64:33356) 2024-04-01T00:14:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.632+0000 7fb307fff640 1 -- 172.21.15.64:0/3338001735 learned_addr learned my addr 172.21.15.64:0/3338001735 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb306ffd640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb3080b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873310413 0 0) 0x7fb300001110 con 0x7fb3080d0900 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2f0003520 con 0x7fb3080d0900 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069843552 0 0) 0x7fb3080d2880 con 0x7fb308161a30 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb300001110 con 0x7fb308161a30 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2698273700 0 0) 0x7fb2f0003520 con 0x7fb3080d0900 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 msgr2=0x7fb3080b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb3080b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 >> v1:172.21.15.100:6789/0 conn(0x7fb308161a30 legacy=0x7fb3080b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3080d2b20 con 0x7fb3080d0900 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fb3000017c0 con 0x7fb3080d0900 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb306ffd640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb3080b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 >> v1:172.21.15.64:6789/0 conn(0x7fb3080d0900 legacy=0x7fb3080cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb2f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb2f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb3080cef20 con 0x7fb2f0003b70 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f000ba70 con 0x7fb2f0004800 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f000bc40 con 0x7fb3080d4a40 2024-04-01T00:14:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb2f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb306ffd640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb307fff640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb2f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 -- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 msgr2=0x7fb2f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080d4a40 0x7fb2f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 -- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0004800 msgr2=0x7fb2f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0004800 0x7fb2f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.633+0000 7fb3077fe640 1 -- 172.21.15.64:0/3338001735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0003520 con 0x7fb2f0003b70 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.634+0000 7fb3077fe640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb2f0003f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8001fa0 tx=0x7fb2f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455d7a82e2b77ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.634+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8001bf0 con 0x7fb2f0003b70 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.634+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f8003b70 con 0x7fb2f0003b70 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.634+0000 7fb3067fc640 1 -- 172.21.15.64:0/3338001735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80050d0 con 0x7fb2f0003b70 2024-04-01T00:14:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 -- 172.21.15.64:0/3338001735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 msgr2=0x7fb2f0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/3338001735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb2f0003f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8001fa0 tx=0x7fb2f0005830 comp rx=0 tx=0).stop 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb307fff640 1 -- 172.21.15.64:0/3338001735 reap_dead start 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 -- 172.21.15.64:0/3338001735 shutdown_connections 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 -- 172.21.15.64:0/3338001735 >> 172.21.15.64:0/3338001735 conn(0x7fb30809d250 msgr2=0x7fb3080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 -- 172.21.15.64:0/3338001735 shutdown_connections 2024-04-01T00:14:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.635+0000 7fb30e1c2640 1 -- 172.21.15.64:0/3338001735 wait complete. 2024-04-01T00:14:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.636+0000 7fb30e1c2640 1 Processor -- start 2024-04-01T00:14:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.636+0000 7fb30e1c2640 1 -- start start 2024-04-01T00:14:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 0x7fb3081f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30815ab90 0x7fb30815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb3080b3740 con 0x7fb2f0003b70 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3080ceb40 con 0x7fb30815ab90 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb30e1c2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb3080b38c0 con 0x7fb30815dbd0 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58242/0 (socket says 172.21.15.64:58242) 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 -- 172.21.15.64:0/2042352037 learned_addr learned my addr 172.21.15.64:0/2042352037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb306ffd640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 0x7fb3081f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb307fff640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30815ab90 0x7fb30815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 -- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 msgr2=0x7fb3081f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 0x7fb3081f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 -- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30815ab90 msgr2=0x7fb30815b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30815ab90 0x7fb30815b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb3077fe640 1 -- 172.21.15.64:0/2042352037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30815e340 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.637+0000 7fb306ffd640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 0x7fb3081f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb3077fe640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8005040 tx=0x7fb2f8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8041030 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f8003970 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30815e620 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8039040 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.638+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3081f4710 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.639+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3080681a0 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.639+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb2f8044070 con 0x7fb2f0003b70 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.639+0000 7fb2effff640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 0x7fb2f406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.640+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fb2f80abc30 con 0x7fb2f0003b70 2024-04-01T00:14:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.640+0000 7fb306ffd640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 0x7fb2f406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.640+0000 7fb306ffd640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 0x7fb2f406de80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb3080c7820 tx=0x7fb2fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.644+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb2f804b070 con 0x7fb2f0003b70 2024-04-01T00:14:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:35.867+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb308072c50 con 0x7fb2f0003b70 2024-04-01T00:14:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.516+0000 7fb2effff640 1 -- 172.21.15.64:0/2042352037 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fb2f8033e20 con 0x7fb2f0003b70 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 msgr2=0x7fb2f406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 0x7fb2f406de80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb3080c7820 tx=0x7fb2fc008040 comp rx=0 tx=0).stop 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 msgr2=0x7fb308160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8005040 tx=0x7fb2f8001650 comp rx=0 tx=0).stop 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 shutdown_connections 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2f406b9c0 0x7fb2f406de80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30815ab90 0x7fb30815b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb30815dbd0 0x7fb3081f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 --2- 172.21.15.64:0/2042352037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2f0003b70 0x7fb308160240 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 >> 172.21.15.64:0/2042352037 conn(0x7fb30809d250 msgr2=0x7fb3080d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 shutdown_connections 2024-04-01T00:14:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.519+0000 7fb30e1c2640 1 -- 172.21.15.64:0/2042352037 wait complete. 2024-04-01T00:14:36.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-01T00:14:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 Processor -- start 2024-04-01T00:14:36.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 -- start start 2024-04-01T00:14:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8e4161b30 0x7ff8e40d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e40cfa50 con 0x7ff8e4161750 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff8e40b3140 con 0x7ff8e4161b30 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.645+0000 7ff8e9c8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e40cfcd0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8e4161b30 0x7ff8e40d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:46448/0 (socket says 172.21.15.64:46448) 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e37fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff8e4161750 0x7ff8e40d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:49158/0 (socket says 172.21.15.64:49158) 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e3fff640 1 -- 172.21.15.64:0/1147007700 learned_addr learned my addr 172.21.15.64:0/1147007700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e2ffd640 1 -- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 msgr2=0x7ff8e40d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e2ffd640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e2ffd640 1 -- 172.21.15.64:0/1147007700 >> v1:172.21.15.100:6789/0 conn(0x7ff8e4161750 legacy=0x7ff8e40d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e2ffd640 1 -- 172.21.15.64:0/1147007700 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e40cff90 con 0x7ff8e4161b30 2024-04-01T00:14:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e3fff640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.646+0000 7ff8e27fc640 1 -- 172.21.15.64:0/1147007700 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525586420 0 0) 0x7ff8d8001110 con 0x7ff8e4161750 2024-04-01T00:14:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.647+0000 7ff8e2ffd640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161b30 0x7ff8e40d2b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40097a0 tx=0x7ff8d402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c608d91fc5ad3d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.647+0000 7ff8e27fc640 1 -- 172.21.15.64:0/1147007700 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40315a0 con 0x7ff8e4161b30 2024-04-01T00:14:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.647+0000 7ff8e27fc640 1 -- 172.21.15.64:0/1147007700 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d4037070 con 0x7ff8e4161b30 2024-04-01T00:14:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.647+0000 7ff8e27fc640 1 -- 172.21.15.64:0/1147007700 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4031960 con 0x7ff8e4161b30 2024-04-01T00:14:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161b30 msgr2=0x7ff8e40d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161b30 0x7ff8e40d2b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40097a0 tx=0x7ff8d402b840 comp rx=0 tx=0).stop 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/1147007700 shutdown_connections 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161b30 0x7ff8e40d2b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/1147007700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e40d1e30 0x7ff8e40d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/1147007700 >> 172.21.15.64:0/1147007700 conn(0x7ff8e409d210 msgr2=0x7ff8e40d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/1147007700 shutdown_connections 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.648+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/1147007700 wait complete. 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.649+0000 7ff8e9c8f640 1 Processor -- start 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.649+0000 7ff8e9c8f640 1 -- start start 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161750 0x7ff8e40b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e4161b30 0x7ff8e40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e40b3fa0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e40b3730 con 0x7ff8e4161750 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e9c8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e40b4120 con 0x7ff8e4161b30 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58246/0 (socket says 172.21.15.64:58246) 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 -- 172.21.15.64:0/874080549 learned_addr learned my addr 172.21.15.64:0/874080549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e2ffd640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161750 0x7ff8e40b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e3fff640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e4161b30 0x7ff8e40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 -- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e4161b30 msgr2=0x7ff8e40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e4161b30 0x7ff8e40b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 -- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161750 msgr2=0x7ff8e40b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161750 0x7ff8e40b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 -- 172.21.15.64:0/874080549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e41f4aa0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8e37fe640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c780 tx=0x7ff8d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.650+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8018070 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.651+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d80072d0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.651+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e41f4d80 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.651+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80047e0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.651+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e41f5360 con 0x7ff8e40d1e30 2024-04-01T00:14:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.652+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e40681a0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.652+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff8d8004980 con 0x7ff8e40d1e30 2024-04-01T00:14:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.652+0000 7ff8cbfff640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 0x7ff8d006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.653+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7ff8d8089cb0 con 0x7ff8e40d1e30 2024-04-01T00:14:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.653+0000 7ff8e2ffd640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 0x7ff8d006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:36.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.653+0000 7ff8e2ffd640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 0x7ff8d006de50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4036000 tx=0x7ff8d4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:36.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.657+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff8e40cfa50 con 0x7ff8e40d1e30 2024-04-01T00:14:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:36.838+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8e415e700 con 0x7ff8e40d1e30 2024-04-01T00:14:37.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.525+0000 7ff8cbfff640 1 -- 172.21.15.64:0/874080549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7ff8d8004c70 con 0x7ff8e40d1e30 2024-04-01T00:14:37.526 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 msgr2=0x7ff8d006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 0x7ff8d006de50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4036000 tx=0x7ff8d4008030 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 msgr2=0x7ff8e4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c780 tx=0x7ff8d800cc40 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 shutdown_connections 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff8d006b990 0x7ff8d006de50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e4161b30 0x7ff8e40b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e4161750 0x7ff8e40b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.527+0000 7ff8e9c8f640 1 --2- 172.21.15.64:0/874080549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e40d1e30 0x7ff8e4158510 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.528+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 >> 172.21.15.64:0/874080549 conn(0x7ff8e409d210 msgr2=0x7ff8e415f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.528+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 shutdown_connections 2024-04-01T00:14:37.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.528+0000 7ff8e9c8f640 1 -- 172.21.15.64:0/874080549 wait complete. 2024-04-01T00:14:37.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-01T00:14:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.653+0000 7fdaf27e5640 1 Processor -- start 2024-04-01T00:14:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 -- start start 2024-04-01T00:14:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdaec15e300 0x7fdaec0d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaec0d1060 0x7fdaec0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec15e6e0 con 0x7fdaec0b2d60 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fdaec0d1590 con 0x7fdaec0d1060 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf27e5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdaec0d2600 con 0x7fdaec15e300 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf0d82640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaec0d1060 0x7fdaec0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf0d82640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaec0d1060 0x7fdaec0b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58254/0 (socket says 172.21.15.64:58254) 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaeb7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdaec0b2d60 0x7fdaec0d0950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33368/0 (socket says 172.21.15.64:33368) 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.654+0000 7fdaf0d82640 1 -- 172.21.15.64:0/299242794 learned_addr learned my addr 172.21.15.64:0/299242794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaebfff640 1 --2- 172.21.15.64:0/299242794 >> v2:172.21.15.100:3300/0 conn(0x7fdaec15e300 0x7fdaec0d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaf0d82640 1 -- 172.21.15.64:0/299242794 >> v2:172.21.15.100:3300/0 conn(0x7fdaec15e300 msgr2=0x7fdaec0d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaf0d82640 1 --2- 172.21.15.64:0/299242794 >> v2:172.21.15.100:3300/0 conn(0x7fdaec15e300 0x7fdaec0d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaf0d82640 1 -- 172.21.15.64:0/299242794 >> v1:172.21.15.64:6789/0 conn(0x7fdaec0b2d60 legacy=0x7fdaec0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaf0d82640 1 -- 172.21.15.64:0/299242794 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec0d28e0 con 0x7fdaec0d1060 2024-04-01T00:14:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.655+0000 7fdaf0d82640 1 --2- 172.21.15.64:0/299242794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec0d1060 0x7fdaec0b0b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdaec068d80 tx=0x7fdae402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1499233b514bf87b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaeaffd640 1 -- 172.21.15.64:0/299242794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae402cb10 con 0x7fdaec0d1060 2024-04-01T00:14:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaeaffd640 1 -- 172.21.15.64:0/299242794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae402ccb0 con 0x7fdaec0d1060 2024-04-01T00:14:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaeaffd640 1 -- 172.21.15.64:0/299242794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae402ce60 con 0x7fdaec0d1060 2024-04-01T00:14:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 -- 172.21.15.64:0/299242794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec0d1060 msgr2=0x7fdaec0b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/299242794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec0d1060 0x7fdaec0b0b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdaec068d80 tx=0x7fdae402bf70 comp rx=0 tx=0).stop 2024-04-01T00:14:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 -- 172.21.15.64:0/299242794 shutdown_connections 2024-04-01T00:14:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/299242794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec0d1060 0x7fdaec0b0b40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/299242794 >> v2:172.21.15.100:3300/0 conn(0x7fdaec15e300 0x7fdaec0d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 -- 172.21.15.64:0/299242794 >> 172.21.15.64:0/299242794 conn(0x7fdaec0d3f80 msgr2=0x7fdaec0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 -- 172.21.15.64:0/299242794 shutdown_connections 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.656+0000 7fdaf27e5640 1 -- 172.21.15.64:0/299242794 wait complete. 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.657+0000 7fdaf27e5640 1 Processor -- start 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 -- start start 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdaec0b2d60 0x7fdaec0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec0d1060 0x7fdaec0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec0b1080 con 0x7fdaec15e300 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec0d1870 con 0x7fdaec0d1060 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf27e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec0b1200 con 0x7fdaec0b2d60 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf0d82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf0d82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58266/0 (socket says 172.21.15.64:58266) 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.658+0000 7fdaf0d82640 1 -- 172.21.15.64:0/2092645636 learned_addr learned my addr 172.21.15.64:0/2092645636 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaeb7fe640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec0d1060 0x7fdaec0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaebfff640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdaec0b2d60 0x7fdaec0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 -- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdaec0b2d60 msgr2=0x7fdaec0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdaec0b2d60 0x7fdaec0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 -- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec0d1060 msgr2=0x7fdaec0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec0d1060 0x7fdaec0ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 -- 172.21.15.64:0/2092645636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec0cda10 con 0x7fdaec15e300 2024-04-01T00:14:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf0d82640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdae4036040 tx=0x7fdae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4043020 con 0x7fdaec15e300 2024-04-01T00:14:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae403e450 con 0x7fdaec15e300 2024-04-01T00:14:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec0cdcf0 con 0x7fdaec15e300 2024-04-01T00:14:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae402ce60 con 0x7fdaec15e300 2024-04-01T00:14:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.659+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec0b6c40 con 0x7fdaec15e300 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.660+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec0b15c0 con 0x7fdaec15e300 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.660+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fdae4046020 con 0x7fdaec15e300 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.661+0000 7fdae8ff9640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 0x7fdad806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.661+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7fdae40acb40 con 0x7fdaec15e300 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.661+0000 7fdaebfff640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 0x7fdad806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.661+0000 7fdaebfff640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 0x7fdad806de80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.665+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fdaec0b15c0 con 0x7fdaec15e300 2024-04-01T00:14:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:37.855+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdaec0681a0 con 0x7fdaec15e300 2024-04-01T00:14:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.526+0000 7fdae8ff9640 1 -- 172.21.15.64:0/2092645636 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7fdae404d070 con 0x7fdaec15e300 2024-04-01T00:14:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-04-01T00:14:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 msgr2=0x7fdad806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 0x7fdad806de80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0008040 comp rx=0 tx=0).stop 2024-04-01T00:14:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 msgr2=0x7fdaec160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdae4036040 tx=0x7fdae4004a40 comp rx=0 tx=0).stop 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 shutdown_connections 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdad806b9c0 0x7fdad806de80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec15e300 0x7fdaec160a80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec0d1060 0x7fdaec0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 --2- 172.21.15.64:0/2092645636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdaec0b2d60 0x7fdaec0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.530+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 >> 172.21.15.64:0/2092645636 conn(0x7fdaec0d3f80 msgr2=0x7fdaec06c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.531+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 shutdown_connections 2024-04-01T00:14:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.531+0000 7fdaf27e5640 1 -- 172.21.15.64:0/2092645636 wait complete. 2024-04-01T00:14:38.549 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-01T00:14:38.549 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-01T00:14:38.549 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-01T00:14:38.549 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:38.549 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-01T00:14:38.549 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-01T00:14:38.550 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-01T00:14:38.550 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:38.550 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-01T00:14:38.550 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-01T00:14:38.550 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-01T00:14:38.550 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:38.550 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-01T00:14:38.551 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-01T00:14:38.551 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-01T00:14:38.551 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T00:14:38.560 INFO:tasks.ceph.mds.a:Stopped 2024-04-01T00:14:38.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-01T00:14:38.561 INFO:tasks.ceph.mds.c:Stopped 2024-04-01T00:14:38.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-01T00:14:38.562 INFO:tasks.ceph.mds.d:Stopped 2024-04-01T00:14:38.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-01T00:14:38.563 INFO:tasks.ceph.mds.b:Stopped 2024-04-01T00:14:38.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-01T00:14:38.602 INFO:tasks.ceph.mds.a:Started 2024-04-01T00:14:38.602 DEBUG:teuthology.parallel:result is None 2024-04-01T00:14:38.603 INFO:tasks.ceph.mds.c:Started 2024-04-01T00:14:38.603 DEBUG:teuthology.parallel:result is None 2024-04-01T00:14:38.605 INFO:tasks.ceph.mds.d:Started 2024-04-01T00:14:38.605 DEBUG:teuthology.parallel:result is None 2024-04-01T00:14:38.606 INFO:tasks.ceph.mds.b:Started 2024-04-01T00:14:38.606 DEBUG:teuthology.parallel:result is None 2024-04-01T00:14:38.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-01T00:14:38.682 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-04-01T00:14:38.682 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-04-01T00:14:38.685 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-01T00:14:38.685 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-01T00:14:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.714+0000 7f21f2119640 1 Processor -- start 2024-04-01T00:14:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.714+0000 7f21f2119640 1 -- start start 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21f2119640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21ec1618f0 0x7f21ec0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21f2119640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21ec0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21f2119640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0d3050 con 0x7f21ec0b4f20 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21f2119640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0d2d70 con 0x7f21ec1618f0 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21f2119640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0d1190 con 0x7f21ec0b3a40 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21eaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f21ec0b4f20 0x7f21ec0d2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33372/0 (socket says 172.21.15.64:33372) 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21eaffd640 1 -- 172.21.15.64:0/2662764967 learned_addr learned my addr 172.21.15.64:0/2662764967 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> v2:172.21.15.100:3300/0 conn(0x7f21ec1618f0 0x7f21ec0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21ebfff640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21ec0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.715+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594367342 0 0) 0x7f21ec0d3050 con 0x7f21ec0b4f20 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21d4003520 con 0x7f21ec0b4f20 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 698232924 0 0) 0x7f21d4003520 con 0x7f21ec0b4f20 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 msgr2=0x7f21ec0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21ec0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 >> v2:172.21.15.100:3300/0 conn(0x7f21ec1618f0 msgr2=0x7f21ec0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 --2- 172.21.15.64:0/2662764967 >> v2:172.21.15.100:3300/0 conn(0x7f21ec1618f0 0x7f21ec0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0d1470 con 0x7f21ec0b4f20 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> v2:172.21.15.100:3300/0 conn(0x7f21ec1618f0 0x7f21ec0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f21dc001670 con 0x7f21ec0b4f20 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 >> v1:172.21.15.64:6789/0 conn(0x7f21ec0b4f20 legacy=0x7f21ec0d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d40039b0 0x7f21d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d4004800 0x7f21d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0d2d70 con 0x7f21d40039b0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0d1190 con 0x7f21d4004800 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f21d400b9c0 con 0x7f21ec0b3a40 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d40039b0 0x7f21d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eaffd640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d4004800 0x7f21d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21ebfff640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 -- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 msgr2=0x7f21d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21ec0b3a40 0x7f21d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 -- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d4004800 msgr2=0x7f21d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d4004800 0x7f21d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.716+0000 7f21eb7fe640 1 -- 172.21.15.64:0/2662764967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d4003520 con 0x7f21d40039b0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.717+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d40039b0 0x7f21d4003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009c90 tx=0x7f21d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f942aa40aa684b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.717+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0039070 con 0x7f21d40039b0 2024-04-01T00:14:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.717+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e002c980 con 0x7f21d40039b0 2024-04-01T00:14:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.717+0000 7f21ea7fc640 1 -- 172.21.15.64:0/2662764967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0009860 con 0x7f21d40039b0 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 -- 172.21.15.64:0/2662764967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d40039b0 msgr2=0x7f21d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 --2- 172.21.15.64:0/2662764967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d40039b0 0x7f21d4003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009c90 tx=0x7f21d4005830 comp rx=0 tx=0).stop 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21ebfff640 1 -- 172.21.15.64:0/2662764967 reap_dead start 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 -- 172.21.15.64:0/2662764967 shutdown_connections 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 -- 172.21.15.64:0/2662764967 >> 172.21.15.64:0/2662764967 conn(0x7f21ec0d3f60 msgr2=0x7f21ec0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 -- 172.21.15.64:0/2662764967 shutdown_connections 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.718+0000 7f21f2119640 1 -- 172.21.15.64:0/2662764967 wait complete. 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.719+0000 7f21f2119640 1 Processor -- start 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.719+0000 7f21f2119640 1 -- start start 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 0x7f21ec0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec0ccd60 0x7f21ec152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec161cd0 con 0x7f21ec0c9e10 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0d2990 con 0x7f21ec0ccd60 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21f2119640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0cd240 con 0x7f21d40039b0 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58336/0 (socket says 172.21.15.64:58336) 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 -- 172.21.15.64:0/2187906961 learned_addr learned my addr 172.21.15.64:0/2187906961 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 0x7f21ec0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21eaffd640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec0ccd60 0x7f21ec152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 -- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 msgr2=0x7f21ec0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 0x7f21ec0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 -- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec0ccd60 msgr2=0x7f21ec152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec0ccd60 0x7f21ec152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 -- 172.21.15.64:0/2187906961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0cd520 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 0x7f21ec0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21ebfff640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21ec067ef0 tx=0x7f21e400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.720+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400fd10 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.721+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4015430 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.721+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec0c7b30 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.721+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4015820 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.721+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec0cd800 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.722+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec0681a0 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.722+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f21e40159c0 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.722+0000 7f21d3fff640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 0x7f21d806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.723+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f21e408b220 con 0x7f21ec0c9e10 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.723+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 0x7f21d806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.723+0000 7f21eb7fe640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 0x7f21d806de80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21e0036000 tx=0x7f21e0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:38.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.727+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f21e4019030 con 0x7f21ec0c9e10 2024-04-01T00:14:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:38.928+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f21ec0b3650 con 0x7f21ec0c9e10 2024-04-01T00:14:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.540+0000 7f21d3fff640 1 -- 172.21.15.64:0/2187906961 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f21e4010040 con 0x7f21ec0c9e10 2024-04-01T00:14:39.541 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2024-04-01T00:14:39.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 msgr2=0x7f21d806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 0x7f21d806de80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21e0036000 tx=0x7f21e0030000 comp rx=0 tx=0).stop 2024-04-01T00:14:39.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 msgr2=0x7f21ec0ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21ec067ef0 tx=0x7f21e400f1b0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 shutdown_connections 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f21d806b9c0 0x7f21d806de80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21ec0c9e10 0x7f21ec0ca2a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec0ccd60 0x7f21ec152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 --2- 172.21.15.64:0/2187906961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40039b0 0x7f21ec0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 >> 172.21.15.64:0/2187906961 conn(0x7f21ec0d3f60 msgr2=0x7f21ec0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 shutdown_connections 2024-04-01T00:14:39.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.543+0000 7f21f2119640 1 -- 172.21.15.64:0/2187906961 wait complete. 2024-04-01T00:14:39.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-01T00:14:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.662+0000 7f5c723f6640 1 Processor -- start 2024-04-01T00:14:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.662+0000 7f5c723f6640 1 -- start start 2024-04-01T00:14:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c723f6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c723f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c723f6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c0b3cb0 con 0x7f5c6c0b3820 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c723f6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c0d2ba0 con 0x7f5c6c1619e0 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c723f6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c0d2e80 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c6bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c6bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:44040/0 (socket says 172.21.15.64:44040) 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c6b7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5c6c0b3820 0x7f5c6c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:49184/0 (socket says 172.21.15.64:49184) 2024-04-01T00:14:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:46540/0 (socket says 172.21.15.64:46540) 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c6bfff640 1 -- 172.21.15.64:0/3593852806 learned_addr learned my addr 172.21.15.64:0/3593852806 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 -- 172.21.15.64:0/3593852806 >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 msgr2=0x7f5c6c0d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 --2- 172.21.15.64:0/3593852806 >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 -- 172.21.15.64:0/3593852806 >> v1:172.21.15.100:6789/0 conn(0x7f5c6c0b3820 legacy=0x7f5c6c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c6affd640 1 -- 172.21.15.64:0/3593852806 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793146625 0 0) 0x7f5c6c0b3cb0 con 0x7f5c6c0b3820 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.663+0000 7f5c70993640 1 -- 172.21.15.64:0/3593852806 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0ceeb0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.664+0000 7f5c6bfff640 1 --2- 172.21.15.64:0/3593852806 >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.664+0000 7f5c70993640 1 --2- 172.21.15.64:0/3593852806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9006f41e30f049f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c6affd640 1 -- 172.21.15.64:0/3593852806 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64039070 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c6affd640 1 -- 172.21.15.64:0/3593852806 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6402cbc0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c6affd640 1 -- 172.21.15.64:0/3593852806 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64034440 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 -- 172.21.15.64:0/3593852806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 msgr2=0x7f5c6c1620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 --2- 172.21.15.64:0/3593852806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6402be10 comp rx=0 tx=0).stop 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 -- 172.21.15.64:0/3593852806 shutdown_connections 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 --2- 172.21.15.64:0/3593852806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c0cfae0 0x7f5c6c1620a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 --2- 172.21.15.64:0/3593852806 >> v2:172.21.15.100:3300/0 conn(0x7f5c6c1619e0 0x7f5c6c0d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 -- 172.21.15.64:0/3593852806 >> 172.21.15.64:0/3593852806 conn(0x7f5c6c09d290 msgr2=0x7f5c6c0d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 -- 172.21.15.64:0/3593852806 shutdown_connections 2024-04-01T00:14:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.665+0000 7f5c723f6640 1 -- 172.21.15.64:0/3593852806 wait complete. 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.666+0000 7f5c723f6640 1 Processor -- start 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.666+0000 7f5c723f6640 1 -- start start 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0b3820 0x7f5c6c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c1619e0 0x7f5c6c1f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0b31b0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0d2890 con 0x7f5c6c0b3820 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c723f6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c0ceb10 con 0x7f5c6c1619e0 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c6b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c6b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58340/0 (socket says 172.21.15.64:58340) 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c6b7fe640 1 -- 172.21.15.64:0/284194287 learned_addr learned my addr 172.21.15.64:0/284194287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c6bfff640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0b3820 0x7f5c6c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.667+0000 7f5c70993640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c1619e0 0x7f5c6c1f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 -- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c1619e0 msgr2=0x7f5c6c1f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c1619e0 0x7f5c6c1f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 -- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0b3820 msgr2=0x7f5c6c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0b3820 0x7f5c6c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 -- 172.21.15.64:0/284194287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0cdff0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c6b7fe640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000c4d0 tx=0x7f5c6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60018070 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c60007500 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c0c8810 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004a20 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.668+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c0c8dd0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.669+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c0681a0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.669+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f5c60004bc0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.670+0000 7f5c68ff9640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 0x7f5c5806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.670+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5c60089730 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.670+0000 7f5c6bfff640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 0x7f5c5806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.671+0000 7f5c6bfff640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 0x7f5c5806de50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006ca0 tx=0x7f5c5c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:39.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.674+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f5c60057fc0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.929+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5c6c0b43c0 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.929+0000 7f5c68ff9640 1 -- 172.21.15.64:0/284194287 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f5c6005c060 con 0x7f5c6c0cfae0 2024-04-01T00:14:39.930 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAA/Almpy5fMBAACQ2Bc0pp2abrcKUna5lLhg==", 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/AlmRpEEARAArkEoLwuviMjZnN98I52POg==", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2024-04-01T00:14:39.932 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAA/Alm5LCsNxAAeCbeyTM3Iye2JyFExe+KEw==", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/Almw7deCBAAu7fj48GD+FcAaC8jknSJjQ==", 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAE/AlmMBLUKhAAY/UsFVVr2VY84mszKGbiBg==", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.934 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAH/AlmJcyLKhAApWlXHrsyypZecQzPJ+PWIw==", 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAK/AlmlGHJKBAAQxxO8RKEod66G8zQUGC3Wg==", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAN/AlmX/hdJxAAtY4W+XjlHbJX6UlYGS52nA==", 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.936 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAT/AlmZg3hKxAAmuuwA+m4et4Dgmjb16rnFA==", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAW/AlmS+QyKhAA9UEyBinWotqBvdIuBDvAFA==", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAZ/AlmQKahKBAARv4D/FmeHLlgwfwRogi16A==", 2024-04-01T00:14:39.938 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAc/AlmDP/AJhAAXuHiBi9ziGwyJ/3b1R8cKg==", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/AlmQSrFDBAAIkPmgw3eT122JxbfaL7NsQ==", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/AlmSVzxFRAAOssJtGlVfNpjHcwnGszLWg==", 2024-04-01T00:14:39.940 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/Almw3ODEhAAqc3CoCnjNcAllG+5kTidqQ==", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-04-01T00:14:39.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAB/AlmUn25GxAADK4DgCw2Bxp/M3c/WWUeWg==", 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.942 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAA/AlmpPqxExAA72Eit4inwHsBJbOT+zZ2jw==", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/AlmXoVsHxAAAw+LSOUsUxLogidqZB2GfA==", 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/AlmjI1sHxAAPi8+29uh06YNZJGEIGB1xw==", 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/AlmdZRsHxAA/MOVKrC0LNjtXCEHRIMgZQ==", 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2024-04-01T00:14:39.944 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/Alm0ZpsHxAAhfflnYdXSFTBQ/D90i1UNg==", 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/AlmYKFsHxAAM2Cg5SOA75pcDWqiHcMM6Q==", 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAi/AlmUKhsHxAAkn6U5heWvrgoRrapmFgMyw==", 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAA/Alm29f6LBAAgGa1IoQeIWAwVGClRmU5/A==", 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-04-01T00:14:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAA/AlmT00PKRAATdhTPJ3xp/s5utlvwwbHnw==", 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-04-01T00:14:39.947 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.931+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 msgr2=0x7f5c5806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.931+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 0x7f5c5806de50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006ca0 tx=0x7f5c5c009240 comp rx=0 tx=0).stop 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.931+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 msgr2=0x7f5c6c153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.931+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000c4d0 tx=0x7f5c6000c990 comp rx=0 tx=0).stop 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 shutdown_connections 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5c5806b990 0x7f5c5806de50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c6c1619e0 0x7f5c6c1f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c6c0cfae0 0x7f5c6c153ca0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 --2- 172.21.15.64:0/284194287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0b3820 0x7f5c6c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 >> 172.21.15.64:0/284194287 conn(0x7f5c6c09d290 msgr2=0x7f5c6c0d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 shutdown_connections 2024-04-01T00:14:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:39.932+0000 7f5c723f6640 1 -- 172.21.15.64:0/284194287 wait complete. 2024-04-01T00:14:39.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-01T00:14:40.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.052+0000 7fa0abf9a640 1 Processor -- start 2024-04-01T00:14:40.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.052+0000 7fa0abf9a640 1 -- start start 2024-04-01T00:14:40.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0abf9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0a40d4b70 0x7fa0a40cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0abf9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa0a40cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0abf9a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a4161d50 con 0x7fa0a40ca600 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0abf9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40b5020 con 0x7fa0a40d4b70 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0abf9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40b4b40 con 0x7fa0a40d2b90 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a9d36640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa0a40ca600 0x7fa0a40d2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33384/0 (socket says 172.21.15.64:33384) 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a9d36640 1 -- 172.21.15.64:0/4075036378 learned_addr learned my addr 172.21.15.64:0/4075036378 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0aa537640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa0a40cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a9535640 1 --2- 172.21.15.64:0/4075036378 >> v2:172.21.15.100:3300/0 conn(0x7fa0a40d4b70 0x7fa0a40cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035838641 0 0) 0x7fa0a4161d50 con 0x7fa0a40ca600 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa090003520 con 0x7fa0a40ca600 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1133763581 0 0) 0x7fa090003520 con 0x7fa0a40ca600 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 msgr2=0x7fa0a40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa0a40cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 >> v2:172.21.15.100:3300/0 conn(0x7fa0a40d4b70 msgr2=0x7fa0a40cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 --2- 172.21.15.64:0/4075036378 >> v2:172.21.15.100:3300/0 conn(0x7fa0a40d4b70 0x7fa0a40cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40d1e90 con 0x7fa0a40ca600 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fa0a0001690 con 0x7fa0a40ca600 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 >> v1:172.21.15.64:6789/0 conn(0x7fa0a40ca600 legacy=0x7fa0a40d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.053+0000 7fa0a8d34640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a8d34640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa090004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a8d34640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40b5020 con 0x7fa0900039b0 2024-04-01T00:14:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40b4b40 con 0x7fa090004d40 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa09000b9c0 con 0x7fa0a40d2b90 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9535640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa090004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0aa537640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 -- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 msgr2=0x7fa090004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a40d2b90 0x7fa090004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 -- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa090004d40 msgr2=0x7fa0900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0a9d36640 1 -- 172.21.15.64:0/4075036378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa090003520 con 0x7fa0900039b0 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.054+0000 7fa0aa537640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c240 tx=0x7fa090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a7231511825e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0007d70 con 0x7fa0900039b0 2024-04-01T00:14:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0030070 con 0x7fa0900039b0 2024-04-01T00:14:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0a8d34640 1 -- 172.21.15.64:0/4075036378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a002cd40 con 0x7fa0900039b0 2024-04-01T00:14:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 -- 172.21.15.64:0/4075036378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 msgr2=0x7fa090003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/4075036378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c240 tx=0x7fa090005830 comp rx=0 tx=0).stop 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0aa537640 1 -- 172.21.15.64:0/4075036378 reap_dead start 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 -- 172.21.15.64:0/4075036378 shutdown_connections 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 -- 172.21.15.64:0/4075036378 >> 172.21.15.64:0/4075036378 conn(0x7fa0a40d3ea0 msgr2=0x7fa0a40d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 -- 172.21.15.64:0/4075036378 shutdown_connections 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.055+0000 7fa0abf9a640 1 -- 172.21.15.64:0/4075036378 wait complete. 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 Processor -- start 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 -- start start 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a40cd630 0x7fa0a40b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a41571e0 0x7fa0a4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d10a0 con 0x7fa0900039b0 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d0f20 con 0x7fa0a40cd630 2024-04-01T00:14:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0abf9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40d1220 con 0x7fa0a41571e0 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0a9d36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0a9d36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58354/0 (socket says 172.21.15.64:58354) 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.057+0000 7fa0a9d36640 1 -- 172.21.15.64:0/2717610618 learned_addr learned my addr 172.21.15.64:0/2717610618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9535640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a40cd630 0x7fa0a40b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0aa537640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a41571e0 0x7fa0a4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 -- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a41571e0 msgr2=0x7fa0a4157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a41571e0 0x7fa0a4157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 -- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a40cd630 msgr2=0x7fa0a40b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a40cd630 0x7fa0a40b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 -- 172.21.15.64:0/2717610618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a41f4d40 con 0x7fa0900039b0 2024-04-01T00:14:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0a9d36640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c360 tx=0x7fa0a0001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0045070 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0004bf0 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a40cc590 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0001cb0 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.058+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a40ccb50 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.059+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a40681a0 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.059+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fa0a0007070 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.060+0000 7fa09a7fc640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 0x7fa08806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.060+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa0a00ab690 con 0x7fa0900039b0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.060+0000 7fa0a9535640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 0x7fa08806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.061+0000 7fa0a9535640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 0x7fa08806de50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d4fa0 tx=0x7fa094008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.064+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fa0a41f4d40 con 0x7fa0900039b0 2024-04-01T00:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.315+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fa0a40d2260 con 0x7fa0900039b0 2024-04-01T00:14:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.317+0000 7fa09a7fc640 1 -- 172.21.15.64:0/2717610618 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fa0a007a6d0 con 0x7fa0900039b0 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 msgr2=0x7fa08806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 0x7fa08806de50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40d4fa0 tx=0x7fa094008040 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 msgr2=0x7fa0a41f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c360 tx=0x7fa0a0001a60 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 shutdown_connections 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fa08806b990 0x7fa08806de50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a41571e0 0x7fa0a4157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a40cd630 0x7fa0a40b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 --2- 172.21.15.64:0/2717610618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0900039b0 0x7fa0a41f4800 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 >> 172.21.15.64:0/2717610618 conn(0x7fa0a40d3ea0 msgr2=0x7fa0a406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:40.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 shutdown_connections 2024-04-01T00:14:40.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.319+0000 7fa0abf9a640 1 -- 172.21.15.64:0/2717610618 wait complete. 2024-04-01T00:14:40.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-01T00:14:40.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.433+0000 7fbb9c735640 1 Processor -- start 2024-04-01T00:14:40.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.433+0000 7fbb9c735640 1 -- start start 2024-04-01T00:14:40.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9c735640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9c735640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9c735640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb940b0570 con 0x7fbb940724f0 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9c735640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9406ba60 con 0x7fbb9406b600 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9c735640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbb94072d10 con 0x7fbb9415e120 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9a4d1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb99cd0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9a4d1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:46578/0 (socket says 172.21.15.64:46578) 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/859730013 learned_addr learned my addr 172.21.15.64:0/859730013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.434+0000 7fbb99cd0640 1 --2- 172.21.15.64:0/859730013 >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:44076/0 (socket says 172.21.15.64:44076) 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/859730013 >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 msgr2=0x7fbb94071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb9a4d1640 1 --2- 172.21.15.64:0/859730013 >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/859730013 >> v1:172.21.15.100:6789/0 conn(0x7fbb940724f0 legacy=0x7fbb94161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/859730013 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb940c6040 con 0x7fbb9415e120 2024-04-01T00:14:40.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb99cd0640 1 --2- 172.21.15.64:0/859730013 >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.435+0000 7fbb9a4d1640 1 --2- 172.21.15.64:0/859730013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009870 tx=0x7fbb9002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2083fcab70732f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb994cf640 1 -- 172.21.15.64:0/859730013 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90034070 con 0x7fbb9415e120 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb994cf640 1 -- 172.21.15.64:0/859730013 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb90030840 con 0x7fbb9415e120 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb994cf640 1 -- 172.21.15.64:0/859730013 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90030c10 con 0x7fbb9415e120 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 -- 172.21.15.64:0/859730013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 msgr2=0x7fbb9406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 --2- 172.21.15.64:0/859730013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009870 tx=0x7fbb9002b980 comp rx=0 tx=0).stop 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 -- 172.21.15.64:0/859730013 shutdown_connections 2024-04-01T00:14:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 --2- 172.21.15.64:0/859730013 >> v2:172.21.15.100:3300/0 conn(0x7fbb9406b600 0x7fbb94071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 --2- 172.21.15.64:0/859730013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9415e120 0x7fbb9406b0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 -- 172.21.15.64:0/859730013 >> 172.21.15.64:0/859730013 conn(0x7fbb940d0ba0 msgr2=0x7fbb940d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 -- 172.21.15.64:0/859730013 shutdown_connections 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.436+0000 7fbb9c735640 1 -- 172.21.15.64:0/859730013 wait complete. 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.437+0000 7fbb9c735640 1 Processor -- start 2024-04-01T00:14:40.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.437+0000 7fbb9c735640 1 -- start start 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb940724f0 0x7fbb941e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 0x7fbb941484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb940c68e0 con 0x7fbb9406b600 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb940c6600 con 0x7fbb9415e120 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9c735640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbb940cc2b0 con 0x7fbb940724f0 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9a4d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9a4d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58366/0 (socket says 172.21.15.64:58366) 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.438+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/1493743444 learned_addr learned my addr 172.21.15.64:0/1493743444 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb99cd0640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb940724f0 0x7fbb941e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9acd2640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 0x7fbb941484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb940724f0 msgr2=0x7fbb941e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb940724f0 0x7fbb941e9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 msgr2=0x7fbb941484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 0x7fbb941484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 -- 172.21.15.64:0/1493743444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb94161610 con 0x7fbb9406b600 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9acd2640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 0x7fbb941484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9a4d1640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb90000c00 tx=0x7fbb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90034040 con 0x7fbb9406b600 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb90039d90 con 0x7fbb9406b600 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90043630 con 0x7fbb9406b600 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941e96a0 con 0x7fbb9406b600 2024-04-01T00:14:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.439+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb940b3ba0 con 0x7fbb9406b600 2024-04-01T00:14:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.440+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb940681a0 con 0x7fbb9406b600 2024-04-01T00:14:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.441+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fbb9004a060 con 0x7fbb9406b600 2024-04-01T00:14:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.441+0000 7fbb8affd640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 0x7fbb7806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.441+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbb900ac3e0 con 0x7fbb9406b600 2024-04-01T00:14:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.441+0000 7fbb99cd0640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 0x7fbb7806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.442+0000 7fbb99cd0640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 0x7fbb7806de50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009560 tx=0x7fbb84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.446+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fbb9003e070 con 0x7fbb9406b600 2024-04-01T00:14:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.698+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fbb940d19d0 con 0x7fbb9406b600 2024-04-01T00:14:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.702+0000 7fbb8affd640 1 -- 172.21.15.64:0/1493743444 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fbb9002d330 con 0x7fbb9406b600 2024-04-01T00:14:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 msgr2=0x7fbb7806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 0x7fbb7806de50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009560 tx=0x7fbb84008040 comp rx=0 tx=0).stop 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 msgr2=0x7fbb941610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb90000c00 tx=0x7fbb900046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 shutdown_connections 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbb7806b990 0x7fbb7806de50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9415e120 0x7fbb941484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb940724f0 0x7fbb941e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 --2- 172.21.15.64:0/1493743444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9406b600 0x7fbb941610d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 >> 172.21.15.64:0/1493743444 conn(0x7fbb940d0ba0 msgr2=0x7fbb940af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 shutdown_connections 2024-04-01T00:14:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.704+0000 7fbb9c735640 1 -- 172.21.15.64:0/1493743444 wait complete. 2024-04-01T00:14:40.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.818+0000 7fe44937a640 1 Processor -- start 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.818+0000 7fe44937a640 1 -- start start 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.818+0000 7fe44937a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe44415e230 0x7fe444161d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe44937a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe44937a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4440d01a0 con 0x7fe4441627e0 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe44937a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fe4440d0cc0 con 0x7fe44415e230 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe44937a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe44415e610 con 0x7fe4440cc5a0 2024-04-01T00:14:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe44415e230 0x7fe444161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe4437fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe4441627e0 0x7fe4440cca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33396/0 (socket says 172.21.15.64:33396) 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe44415e230 0x7fe444161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58378/0 (socket says 172.21.15.64:58378) 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe4437fe640 1 -- 172.21.15.64:0/3418672367 learned_addr learned my addr 172.21.15.64:0/3418672367 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe43bfff640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 -- 172.21.15.64:0/3418672367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 msgr2=0x7fe4441622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 -- 172.21.15.64:0/3418672367 >> v1:172.21.15.64:6789/0 conn(0x7fe4441627e0 legacy=0x7fe4440cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe442ffd640 1 -- 172.21.15.64:0/3418672367 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440d0420 con 0x7fe44415e230 2024-04-01T00:14:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.819+0000 7fe43bfff640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe442ffd640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe444161d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4340097a0 tx=0x7fe43402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5380165eaeda5a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe4427fc640 1 -- 172.21.15.64:0/3418672367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434035020 con 0x7fe44415e230 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe4427fc640 1 -- 172.21.15.64:0/3418672367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe434031830 con 0x7fe44415e230 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe4427fc640 1 -- 172.21.15.64:0/3418672367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434031c00 con 0x7fe44415e230 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 -- 172.21.15.64:0/3418672367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 msgr2=0x7fe444161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe444161d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4340097a0 tx=0x7fe43402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 -- 172.21.15.64:0/3418672367 shutdown_connections 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 --2- 172.21.15.64:0/3418672367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe444161d60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.820+0000 7fe44937a640 1 -- 172.21.15.64:0/3418672367 >> 172.21.15.64:0/3418672367 conn(0x7fe4440719a0 msgr2=0x7fe4440b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.821+0000 7fe44937a640 1 -- 172.21.15.64:0/3418672367 shutdown_connections 2024-04-01T00:14:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.821+0000 7fe44937a640 1 -- 172.21.15.64:0/3418672367 wait complete. 2024-04-01T00:14:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 Processor -- start 2024-04-01T00:14:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 -- start start 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441627e0 0x7fe4440b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440cbd00 con 0x7fe44415e230 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d09e0 con 0x7fe4441627e0 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe44937a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440cbe80 con 0x7fe4440cc5a0 2024-04-01T00:14:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe43bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe43bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58382/0 (socket says 172.21.15.64:58382) 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.822+0000 7fe43bfff640 1 -- 172.21.15.64:0/2270007531 learned_addr learned my addr 172.21.15.64:0/2270007531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe442ffd640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe4437fe640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441627e0 0x7fe4440b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 -- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 msgr2=0x7fe4441f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 -- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441627e0 msgr2=0x7fe4440b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441627e0 0x7fe4440b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 -- 172.21.15.64:0/2270007531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440b33a0 con 0x7fe44415e230 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe442ffd640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43bfff640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c7f0 tx=0x7fe42c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c015070 con 0x7fe44415e230 2024-04-01T00:14:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c007820 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4440b16a0 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c007c10 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.823+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe444161560 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.824+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440681a0 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.825+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fe42c018070 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.825+0000 7fe43b7fe640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 0x7fe42406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.825+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fe42c08a3b0 con 0x7fe44415e230 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.825+0000 7fe442ffd640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 0x7fe42406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.826+0000 7fe442ffd640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 0x7fe42406de50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe434036000 tx=0x7fe434008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:40.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:40.829+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fe42c0593a0 con 0x7fe44415e230 2024-04-01T00:14:41.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.084+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fe4440af6b0 con 0x7fe44415e230 2024-04-01T00:14:41.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.088+0000 7fe43b7fe640 1 -- 172.21.15.64:0/2270007531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fe42c058ca0 con 0x7fe44415e230 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 msgr2=0x7fe42406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 0x7fe42406de50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe434036000 tx=0x7fe434008030 comp rx=0 tx=0).stop 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 msgr2=0x7fe4440b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c7f0 tx=0x7fe42c00ccb0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 shutdown_connections 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fe42406b990 0x7fe42406de50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441627e0 0x7fe4440b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe44415e230 0x7fe4440b0920 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 --2- 172.21.15.64:0/2270007531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4440cc5a0 0x7fe4441f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 >> 172.21.15.64:0/2270007531 conn(0x7fe4440719a0 msgr2=0x7fe4441563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 shutdown_connections 2024-04-01T00:14:41.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.090+0000 7fe44937a640 1 -- 172.21.15.64:0/2270007531 wait complete. 2024-04-01T00:14:41.102 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-01T00:14:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.204+0000 7f6b1d698640 1 Processor -- start 2024-04-01T00:14:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.204+0000 7f6b1d698640 1 -- start start 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cb00 0x7f6b180cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6b1806cee0 0x7f6b1815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1815de20 0x7f6b18161c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f6b180b1ce0 con 0x7f6b1815de20 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6b180cf4d0 con 0x7f6b1806cee0 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b1d698640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6b180b1310 con 0x7f6b1806cb00 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1815de20 0x7f6b18161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1815de20 0x7f6b18161c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58392/0 (socket says 172.21.15.64:58392) 2024-04-01T00:14:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 -- 172.21.15.64:0/2725119089 learned_addr learned my addr 172.21.15.64:0/2725119089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b16ffd640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cb00 0x7f6b180cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b0ffff640 1 --2- 172.21.15.64:0/2725119089 >> v2:172.21.15.100:3300/0 conn(0x7f6b1806cee0 0x7f6b1815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 -- 172.21.15.64:0/2725119089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cb00 msgr2=0x7f6b180cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cb00 0x7f6b180cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 -- 172.21.15.64:0/2725119089 >> v2:172.21.15.100:3300/0 conn(0x7f6b1806cee0 msgr2=0x7f6b1815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 --2- 172.21.15.64:0/2725119089 >> v2:172.21.15.100:3300/0 conn(0x7f6b1806cee0 0x7f6b1815d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.205+0000 7f6b177fe640 1 -- 172.21.15.64:0/2725119089 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180b1fa0 con 0x7f6b1815de20 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.206+0000 7f6b177fe640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1815de20 0x7f6b18161c30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b180cf760 tx=0x7f6b1002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d61b1d3dc3e3941 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.206+0000 7f6b167fc640 1 -- 172.21.15.64:0/2725119089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002ca70 con 0x7f6b1815de20 2024-04-01T00:14:41.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.206+0000 7f6b167fc640 1 -- 172.21.15.64:0/2725119089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1002cc10 con 0x7f6b1815de20 2024-04-01T00:14:41.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.206+0000 7f6b167fc640 1 -- 172.21.15.64:0/2725119089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b100343f0 con 0x7f6b1815de20 2024-04-01T00:14:41.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 -- 172.21.15.64:0/2725119089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1815de20 msgr2=0x7f6b18161c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1815de20 0x7f6b18161c30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b180cf760 tx=0x7f6b1002bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 -- 172.21.15.64:0/2725119089 shutdown_connections 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1815de20 0x7f6b18161c30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 --2- 172.21.15.64:0/2725119089 >> v2:172.21.15.100:3300/0 conn(0x7f6b1806cee0 0x7f6b1815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 --2- 172.21.15.64:0/2725119089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cb00 0x7f6b180cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 -- 172.21.15.64:0/2725119089 >> 172.21.15.64:0/2725119089 conn(0x7f6b1809a980 msgr2=0x7f6b1809ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 -- 172.21.15.64:0/2725119089 shutdown_connections 2024-04-01T00:14:41.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.207+0000 7f6b1d698640 1 -- 172.21.15.64:0/2725119089 wait complete. 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.208+0000 7f6b1d698640 1 Processor -- start 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.208+0000 7f6b1d698640 1 -- start start 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cee0 0x7f6b180b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 0x7f6b180b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180b1960 con 0x7f6b1806cb00 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180b2560 con 0x7f6b1815de20 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1815e2b0 con 0x7f6b1806cee0 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58400/0 (socket says 172.21.15.64:58400) 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 -- 172.21.15.64:0/3763400215 learned_addr learned my addr 172.21.15.64:0/3763400215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b0ffff640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cee0 0x7f6b180b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b177fe640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 0x7f6b180b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 -- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cee0 msgr2=0x7f6b180b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cee0 0x7f6b180b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 -- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 msgr2=0x7f6b180b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 0x7f6b180b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 -- 172.21.15.64:0/3763400215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180d1a50 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b177fe640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 0x7f6b180b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b16ffd640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800d830 tx=0x7f6b0800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08018030 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0800be10 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1815f260 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080098b0 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.209+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1815f820 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.210+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b180681a0 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.211+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6b0801b070 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.211+0000 7f6b0f7fe640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 0x7f6af806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.211+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6b0808a6f0 con 0x7f6b1806cb00 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.211+0000 7f6b0ffff640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 0x7f6af806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.212+0000 7f6b0ffff640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 0x7f6af806de50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006fd0 tx=0x7f6b00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.215+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6b08059650 con 0x7f6b1806cb00 2024-04-01T00:14:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.466+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f6b18162b00 con 0x7f6b1806cb00 2024-04-01T00:14:41.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.470+0000 7f6b0f7fe640 1 -- 172.21.15.64:0/3763400215 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f6b08058f50 con 0x7f6b1806cb00 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 msgr2=0x7f6af806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 0x7f6af806de50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006fd0 tx=0x7f6b00008040 comp rx=0 tx=0).stop 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 msgr2=0x7f6b18151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800d830 tx=0x7f6b0800dcf0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 shutdown_connections 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6af806b990 0x7f6af806de50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1815de20 0x7f6b180b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b1806cee0 0x7f6b180b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 --2- 172.21.15.64:0/3763400215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1806cb00 0x7f6b18151fe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 >> 172.21.15.64:0/3763400215 conn(0x7f6b1809a980 msgr2=0x7f6b180af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 shutdown_connections 2024-04-01T00:14:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.472+0000 7f6b1d698640 1 -- 172.21.15.64:0/3763400215 wait complete. 2024-04-01T00:14:41.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-01T00:14:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.586+0000 7fbc0d4b5640 1 Processor -- start 2024-04-01T00:14:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.586+0000 7fbc0d4b5640 1 -- start start 2024-04-01T00:14:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc0d4b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbc080cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc0d4b5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc08161d50 con 0x7fbc080d4b70 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc0d4b5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc080b5020 con 0x7fbc080ca600 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc0d4b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbc080b4b40 con 0x7fbc080d2b90 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbbfe7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbc080d4b70 0x7fbc080cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33406/0 (socket says 172.21.15.64:33406) 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbbfe7fc640 1 -- 172.21.15.64:0/3629260943 learned_addr learned my addr 172.21.15.64:0/3629260943 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215933040 0 0) 0x7fbc08161d50 con 0x7fbc080d4b70 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc077fe640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbc080cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf4003520 con 0x7fbc080d4b70 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1134201920 0 0) 0x7fbbf4003520 con 0x7fbc080d4b70 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 msgr2=0x7fbc080cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbc080cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 >> v1:172.21.15.100:6789/0 conn(0x7fbc080ca600 legacy=0x7fbc080d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080d1e90 con 0x7fbc080d4b70 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7fbbf0001690 con 0x7fbc080d4b70 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 >> v1:172.21.15.64:6789/0 conn(0x7fbc080d4b70 legacy=0x7fbc080cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.587+0000 7fbc067fc640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc067fc640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbbf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc067fc640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbbf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080b4b40 con 0x7fbbf4004d40 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf400bb50 con 0x7fbbf40039e0 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf400bd50 con 0x7fbc080d2b90 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbbf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc06ffd640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbbfe7fc640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbbf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 -- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 msgr2=0x7fbbf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc080d2b90 0x7fbbf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 -- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf40039e0 msgr2=0x7fbbf4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.588+0000 7fbc077fe640 1 -- 172.21.15.64:0/3629260943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf4003520 con 0x7fbbf4004d40 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc077fe640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbbf40051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007360 tx=0x7fbbf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9e945469c1e5da server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00039070 con 0x7fbbf4004d40 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0002ccd0 con 0x7fbbf4004d40 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc067fc640 1 -- 172.21.15.64:0/3629260943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00009be0 con 0x7fbbf4004d40 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/3629260943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 msgr2=0x7fbbf40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/3629260943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbbf40051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007360 tx=0x7fbbf4005830 comp rx=0 tx=0).stop 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc077fe640 1 -- 172.21.15.64:0/3629260943 reap_dead start 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/3629260943 shutdown_connections 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/3629260943 >> 172.21.15.64:0/3629260943 conn(0x7fbc080d3ea0 msgr2=0x7fbc080d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/3629260943 shutdown_connections 2024-04-01T00:14:41.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.589+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/3629260943 wait complete. 2024-04-01T00:14:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 Processor -- start 2024-04-01T00:14:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 -- start start 2024-04-01T00:14:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 0x7fbc080b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 0x7fbc0806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080d10a0 con 0x7fbbf4004d40 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080d0f20 con 0x7fbc0806cbb0 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc0d4b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbc080d1220 con 0x7fbc0809f220 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.591+0000 7fbc06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58422/0 (socket says 172.21.15.64:58422) 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 -- 172.21.15.64:0/1828995583 learned_addr learned my addr 172.21.15.64:0/1828995583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc077fe640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 0x7fbc0806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbbfe7fc640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 0x7fbc080b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 -- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 msgr2=0x7fbc080b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 0x7fbc080b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 -- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 msgr2=0x7fbc0806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 0x7fbc0806d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 -- 172.21.15.64:0/1828995583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080b0e50 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc077fe640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 0x7fbc0806d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbbfe7fc640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 0x7fbc080b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc06ffd640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800f820 tx=0x7fbbf800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80050d0 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf8018070 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc080b1130 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8005490 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.592+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc080b16f0 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.593+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0806d570 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.593+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fbbf800d730 con 0x7fbbf4004d40 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.594+0000 7fbbfffff640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 0x7fbbe806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.594+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbbf808a3b0 con 0x7fbbf4004d40 2024-04-01T00:14:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.594+0000 7fbbfe7fc640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 0x7fbbe806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.595+0000 7fbbfe7fc640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 0x7fbbe806de80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0001eb0 tx=0x7fbbf0001de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.598+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fbbf8059310 con 0x7fbbf4004d40 2024-04-01T00:14:41.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.851+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fbc080d2220 con 0x7fbbf4004d40 2024-04-01T00:14:41.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.853+0000 7fbbfffff640 1 -- 172.21.15.64:0/1828995583 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fbbf8058c10 con 0x7fbbf4004d40 2024-04-01T00:14:41.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 msgr2=0x7fbbe806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 0x7fbbe806de80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0001eb0 tx=0x7fbbf0001de0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 msgr2=0x7fbc08157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800f820 tx=0x7fbbf800fce0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 shutdown_connections 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbbe806b9c0 0x7fbbe806de80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0806cbb0 0x7fbc0806d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc0809f220 0x7fbc080b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 --2- 172.21.15.64:0/1828995583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf4004d40 0x7fbc08157500 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 >> 172.21.15.64:0/1828995583 conn(0x7fbc080d3ea0 msgr2=0x7fbc080cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 shutdown_connections 2024-04-01T00:14:41.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.855+0000 7fbc0d4b5640 1 -- 172.21.15.64:0/1828995583 wait complete. 2024-04-01T00:14:41.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-01T00:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.969+0000 7f3153b43640 1 Processor -- start 2024-04-01T00:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 -- start start 2024-04-01T00:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f314c0d4a40 0x7f314c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f314c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314c161e10 con 0x7f314c161a30 2024-04-01T00:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f314c0cef20 con 0x7f314c0d4a40 2024-04-01T00:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.970+0000 7f3153b43640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f314c0d2880 con 0x7f314c0d0900 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31518df640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f314c161a30 0x7f314c0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33418/0 (socket says 172.21.15.64:33418) 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31518df640 1 -- 172.21.15.64:0/3121872177 learned_addr learned my addr 172.21.15.64:0/3121872177 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31510de640 1 --2- 172.21.15.64:0/3121872177 >> v2:172.21.15.100:3300/0 conn(0x7f314c0d4a40 0x7f314c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31520e0640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f314c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154400883 0 0) 0x7f3148001110 con 0x7f314c161a30 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3138003520 con 0x7f314c161a30 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2443998265 0 0) 0x7f3138003520 con 0x7f314c161a30 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 msgr2=0x7f314c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f314c0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 >> v2:172.21.15.100:3300/0 conn(0x7f314c0d4a40 msgr2=0x7f314c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 --2- 172.21.15.64:0/3121872177 >> v2:172.21.15.100:3300/0 conn(0x7f314c0d4a40 0x7f314c0b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c0d2b20 con 0x7f314c161a30 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f3148001840 con 0x7f314c161a30 2024-04-01T00:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 >> v1:172.21.15.64:6789/0 conn(0x7f314c161a30 legacy=0x7f314c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f31380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138003f30 0x7f3138004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138004d40 0x7f31380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0cef20 con 0x7f3138004d40 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0d2880 con 0x7f3138003f30 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f313800b9c0 con 0x7f314c0d0900 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31520e0640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138004d40 0x7f31380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31518df640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f31380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.971+0000 7f31510de640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138003f30 0x7f3138004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 -- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 msgr2=0x7f31380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c0d0900 0x7f31380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 -- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138003f30 msgr2=0x7f3138004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138003f30 0x7f3138004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 -- 172.21.15.64:0/3121872177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3138003520 con 0x7f3138004d40 2024-04-01T00:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31520e0640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138004d40 0x7f31380051c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3144009fc0 tx=0x7f3138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4327789e9c5f883 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.972+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144039070 con 0x7f3138004d40 2024-04-01T00:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314402cc00 con 0x7f3138004d40 2024-04-01T00:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f31508dd640 1 -- 172.21.15.64:0/3121872177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144009c10 con 0x7f3138004d40 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 -- 172.21.15.64:0/3121872177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138004d40 msgr2=0x7f31380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 --2- 172.21.15.64:0/3121872177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3138004d40 0x7f31380051c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3144009fc0 tx=0x7f3138005830 comp rx=0 tx=0).stop 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f31520e0640 1 -- 172.21.15.64:0/3121872177 reap_dead start 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 -- 172.21.15.64:0/3121872177 shutdown_connections 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 -- 172.21.15.64:0/3121872177 >> 172.21.15.64:0/3121872177 conn(0x7f314c09d250 msgr2=0x7f314c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 -- 172.21.15.64:0/3121872177 shutdown_connections 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.973+0000 7f3153b43640 1 -- 172.21.15.64:0/3121872177 wait complete. 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.974+0000 7f3153b43640 1 Processor -- start 2024-04-01T00:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.974+0000 7f3153b43640 1 -- start start 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 0x7f314c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 0x7f314c1f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0d0d90 con 0x7f314c1f4e90 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0ceb40 con 0x7f314c1f44d0 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f3153b43640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f314c0d0f10 con 0x7f314c1545a0 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58436/0 (socket says 172.21.15.64:58436) 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 -- 172.21.15.64:0/2797518388 learned_addr learned my addr 172.21.15.64:0/2797518388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31518df640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 0x7f314c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31510de640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 0x7f314c1f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 -- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 msgr2=0x7f314c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 0x7f314c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 -- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 msgr2=0x7f314c1f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 0x7f314c1f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31520e0640 1 -- 172.21.15.64:0/2797518388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c15d870 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31510de640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 0x7f314c1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.975+0000 7f31518df640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 0x7f314c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f31520e0640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f314400e230 tx=0x7f3144009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144039030 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314403a040 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c154980 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144009c10 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.976+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c0cda90 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.977+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c0b47b0 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.977+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3144045020 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.978+0000 7f31427fc640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 0x7f313006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.978+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f31440ac070 con 0x7f314c1f4e90 2024-04-01T00:14:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.978+0000 7f31518df640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 0x7f313006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.978+0000 7f31518df640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 0x7f313006de50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31480012b0 tx=0x7f3148009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:41.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:41.982+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f314404c020 con 0x7f314c1f4e90 2024-04-01T00:14:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.236+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f314c0681a0 con 0x7f314c1f4e90 2024-04-01T00:14:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.238+0000 7f31427fc640 1 -- 172.21.15.64:0/2797518388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f314c15d870 con 0x7f314c1f4e90 2024-04-01T00:14:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 msgr2=0x7f313006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 0x7f313006de50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31480012b0 tx=0x7f3148009040 comp rx=0 tx=0).stop 2024-04-01T00:14:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 msgr2=0x7f314c1f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f314400e230 tx=0x7f3144009750 comp rx=0 tx=0).stop 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 shutdown_connections 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f313006b990 0x7f313006de50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c1f4e90 0x7f314c1f7b10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314c1f44d0 0x7f314c1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 --2- 172.21.15.64:0/2797518388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314c1545a0 0x7f314c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 >> 172.21.15.64:0/2797518388 conn(0x7f314c09d250 msgr2=0x7f314c0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 shutdown_connections 2024-04-01T00:14:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.240+0000 7f3153b43640 1 -- 172.21.15.64:0/2797518388 wait complete. 2024-04-01T00:14:42.252 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.353+0000 7f68a132c640 1 Processor -- start 2024-04-01T00:14:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.353+0000 7f68a132c640 1 -- start start 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f68a132c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f689c0d2740 0x7f689c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f68a132c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f689c0d1030 0x7f689c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f68a132c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689c0b5070 con 0x7f689c0d1cd0 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f68a132c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f689c0d4e60 con 0x7f689c0d1030 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f68a132c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f689c0b5350 con 0x7f689c0d2740 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f689c0d1030 0x7f689c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f689c0d1cd0 0x7f689c0ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33432/0 (socket says 172.21.15.64:33432) 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689b7fe640 1 -- 172.21.15.64:0/1188274387 learned_addr learned my addr 172.21.15.64:0/1188274387 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689a7fc640 1 -- 172.21.15.64:0/1188274387 >> v2:172.21.15.100:3300/0 conn(0x7f689c0d2740 msgr2=0x7f689c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689a7fc640 1 --2- 172.21.15.64:0/1188274387 >> v2:172.21.15.100:3300/0 conn(0x7f689c0d2740 0x7f689c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689a7fc640 1 -- 172.21.15.64:0/1188274387 >> v1:172.21.15.64:6789/0 conn(0x7f689c0d1cd0 legacy=0x7f689c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f6899ffb640 1 -- 172.21.15.64:0/1188274387 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135734533 0 0) 0x7f689c0b5070 con 0x7f689c0d1cd0 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.354+0000 7f689a7fc640 1 -- 172.21.15.64:0/1188274387 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c0b6de0 con 0x7f689c0d1030 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.355+0000 7f689a7fc640 1 --2- 172.21.15.64:0/1188274387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1030 0x7f689c1619c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f689c0d3e50 tx=0x7f688402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c887d85e7b004500 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.355+0000 7f6899ffb640 1 -- 172.21.15.64:0/1188274387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884035020 con 0x7f689c0d1030 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.355+0000 7f6899ffb640 1 -- 172.21.15.64:0/1188274387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884031830 con 0x7f689c0d1030 2024-04-01T00:14:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.355+0000 7f6899ffb640 1 -- 172.21.15.64:0/1188274387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884031c00 con 0x7f689c0d1030 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 -- 172.21.15.64:0/1188274387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1030 msgr2=0x7f689c1619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 --2- 172.21.15.64:0/1188274387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1030 0x7f689c1619c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f689c0d3e50 tx=0x7f688402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 -- 172.21.15.64:0/1188274387 shutdown_connections 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 --2- 172.21.15.64:0/1188274387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1030 0x7f689c1619c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 --2- 172.21.15.64:0/1188274387 >> v2:172.21.15.100:3300/0 conn(0x7f689c0d2740 0x7f689c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 -- 172.21.15.64:0/1188274387 >> 172.21.15.64:0/1188274387 conn(0x7f689c0d3ef0 msgr2=0x7f689c0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 -- 172.21.15.64:0/1188274387 shutdown_connections 2024-04-01T00:14:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.356+0000 7f68a132c640 1 -- 172.21.15.64:0/1188274387 wait complete. 2024-04-01T00:14:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.357+0000 7f68a132c640 1 Processor -- start 2024-04-01T00:14:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.357+0000 7f68a132c640 1 -- start start 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f689c0d1030 0x7f689c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 0x7f689c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0d4b90 con 0x7f689c0d1cd0 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0d4a10 con 0x7f689c0d2740 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f68a132c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f689c0b8310 con 0x7f689c0d1030 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58454/0 (socket says 172.21.15.64:58454) 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 -- 172.21.15.64:0/3294304265 learned_addr learned my addr 172.21.15.64:0/3294304265 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689affd640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f689c0d1030 0x7f689c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689b7fe640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 0x7f689c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 -- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f689c0d1030 msgr2=0x7f689c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f689c0d1030 0x7f689c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 -- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 msgr2=0x7f689c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 0x7f689c15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689a7fc640 1 -- 172.21.15.64:0/3294304265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c159640 con 0x7f689c0d1cd0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.358+0000 7f689b7fe640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 0x7f689c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f689a7fc640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6884009890 tx=0x7f68840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688403ba80 con 0x7f689c0d1cd0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688403bc40 con 0x7f689c0d1cd0 2024-04-01T00:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840453f0 con 0x7f689c0d1cd0 2024-04-01T00:14:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c159920 con 0x7f689c0d1cd0 2024-04-01T00:14:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.359+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c159ee0 con 0x7f689c0d1cd0 2024-04-01T00:14:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.360+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c0681a0 con 0x7f689c0d1cd0 2024-04-01T00:14:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.360+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6884038020 con 0x7f689c0d1cd0 2024-04-01T00:14:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.361+0000 7f68937fe640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 0x7f687c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.361+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f68840ac340 con 0x7f689c0d1cd0 2024-04-01T00:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.361+0000 7f689affd640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 0x7f687c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.361+0000 7f689affd640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 0x7f687c06de50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f689400b810 tx=0x7f6894008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.365+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6884040070 con 0x7f689c0d1cd0 2024-04-01T00:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.547+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f689c0d22b0 con 0x7f689c0d1cd0 2024-04-01T00:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.550+0000 7f68937fe640 1 -- 172.21.15.64:0/3294304265 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12867 (secure 0 0 0) 0x7f688407b2a0 con 0x7f689c0d1cd0 2024-04-01T00:14:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:42.551 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:39.548408+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 msgr2=0x7f687c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 0x7f687c06de50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f689400b810 tx=0x7f6894008040 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 msgr2=0x7f689c15a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6884009890 tx=0x7f68840046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 shutdown_connections 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f687c06b990 0x7f687c06de50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689c0d2740 0x7f689c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f689c0d1cd0 0x7f689c15a6e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 --2- 172.21.15.64:0/3294304265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f689c0d1030 0x7f689c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 >> 172.21.15.64:0/3294304265 conn(0x7f689c0d3ef0 msgr2=0x7f689c0b7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 shutdown_connections 2024-04-01T00:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.552+0000 7f68a132c640 1 -- 172.21.15.64:0/3294304265 wait complete. 2024-04-01T00:14:42.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T00:14:42.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T00:14:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 Processor -- start 2024-04-01T00:14:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 -- start start 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7ff0080afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0080d1230 con 0x7ff00815c000 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0080b0360 con 0x7ff0080d0da0 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff00e22e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080b0640 con 0x7ff00815c3e0 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.668+0000 7ff0077fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff00815c000 0x7ff0080ad0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33442/0 (socket says 172.21.15.64:33442) 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0077fe640 1 -- 172.21.15.64:0/2890015900 learned_addr learned my addr 172.21.15.64:0/2890015900 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff006ffd640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7ff0080afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2729156064 0 0) 0x7ff0080d1230 con 0x7ff00815c000 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feff4003520 con 0x7ff00815c000 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3788945416 0 0) 0x7feff4003520 con 0x7ff00815c000 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 msgr2=0x7ff0080afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7ff0080afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 >> v1:172.21.15.100:6789/0 conn(0x7ff0080d0da0 legacy=0x7ff0080b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080b0920 con 0x7ff00815c000 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7feff0001670 con 0x7ff00815c000 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 >> v1:172.21.15.64:6789/0 conn(0x7ff00815c000 legacy=0x7ff0080ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff40039e0 0x7feff4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7feff4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feff4004d40 0x7feff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080b0640 con 0x7feff4004d40 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feff400bb50 con 0x7feff40039e0 2024-04-01T00:14:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7feff400bd50 con 0x7ff00815c3e0 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff007fff640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feff4004d40 0x7feff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff0077fe640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff40039e0 0x7feff4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff006ffd640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7feff4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff007fff640 1 -- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 msgr2=0x7feff4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff007fff640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7feff4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff007fff640 1 -- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff40039e0 msgr2=0x7feff4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.669+0000 7ff007fff640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff40039e0 0x7feff4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff007fff640 1 -- 172.21.15.64:0/2890015900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff4003520 con 0x7feff4004d40 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff006ffd640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00815c3e0 0x7feff4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff007fff640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feff4004d40 0x7feff40051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff000001f60 tx=0x7feff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec005fe96933fa36 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000001b30 con 0x7feff4004d40 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff00002d710 con 0x7feff4004d40 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.670+0000 7ff0067fc640 1 -- 172.21.15.64:0/2890015900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00002db00 con 0x7feff4004d40 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 -- 172.21.15.64:0/2890015900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feff4004d40 msgr2=0x7feff40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2890015900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feff4004d40 0x7feff40051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff000001f60 tx=0x7feff4005830 comp rx=0 tx=0).stop 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff007fff640 1 -- 172.21.15.64:0/2890015900 reap_dead start 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 -- 172.21.15.64:0/2890015900 shutdown_connections 2024-04-01T00:14:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 -- 172.21.15.64:0/2890015900 >> 172.21.15.64:0/2890015900 conn(0x7ff008072810 msgr2=0x7ff008072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 -- 172.21.15.64:0/2890015900 shutdown_connections 2024-04-01T00:14:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.671+0000 7ff00e22e640 1 -- 172.21.15.64:0/2890015900 wait complete. 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.672+0000 7ff00e22e640 1 Processor -- start 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.672+0000 7ff00e22e640 1 -- start start 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 0x7ff0081f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00815e2d0 0x7ff0080cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080ad800 con 0x7ff0080cf1e0 2024-04-01T00:14:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080b0ee0 con 0x7ff00815e2d0 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff00e22e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080b1b10 con 0x7feff4004d40 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff007fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff007fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58476/0 (socket says 172.21.15.64:58476) 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff007fff640 1 -- 172.21.15.64:0/2423940688 learned_addr learned my addr 172.21.15.64:0/2423940688 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff006ffd640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00815e2d0 0x7ff0080cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.673+0000 7ff0077fe640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 0x7ff0081f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 -- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 msgr2=0x7ff0081f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 0x7ff0081f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 -- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00815e2d0 msgr2=0x7ff0080cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00815e2d0 0x7ff0080cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 -- 172.21.15.64:0/2423940688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0081f6b20 con 0x7ff0080cf1e0 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff0077fe640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 0x7ff0081f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T00:14:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff007fff640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff000002050 tx=0x7ff0000015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000001b30 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff000045070 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00815e760 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000003d70 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.674+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00815ed20 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.675+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0080681a0 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.675+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff000005070 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.676+0000 7feffffff640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 0x7fefe806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.676+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff0000ac5f0 con 0x7ff0080cf1e0 2024-04-01T00:14:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.676+0000 7ff0077fe640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 0x7fefe806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.677+0000 7ff0077fe640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 0x7fefe806de50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feff00016f0 tx=0x7feff0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.680+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff00007b5e0 con 0x7ff0080cf1e0 2024-04-01T00:14:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.864+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0080c7c00 con 0x7ff0080cf1e0 2024-04-01T00:14:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.865+0000 7feffffff640 1 -- 172.21.15.64:0/2423940688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7ff00007aee0 con 0x7ff0080cf1e0 2024-04-01T00:14:42.866 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 msgr2=0x7fefe806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 0x7fefe806de50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feff00016f0 tx=0x7feff0009040 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 msgr2=0x7ff0080cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff000002050 tx=0x7ff0000015b0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 shutdown_connections 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fefe806b990 0x7fefe806de50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0080cf1e0 0x7ff0080cf690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00815e2d0 0x7ff0080cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 --2- 172.21.15.64:0/2423940688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff4004d40 0x7ff0081f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 >> 172.21.15.64:0/2423940688 conn(0x7ff008072810 msgr2=0x7ff0080d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 shutdown_connections 2024-04-01T00:14:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.867+0000 7ff00e22e640 1 -- 172.21.15.64:0/2423940688 wait complete. 2024-04-01T00:14:42.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-01T00:14:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.979+0000 7f6aaf556640 1 Processor -- start 2024-04-01T00:14:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.979+0000 7f6aaf556640 1 -- start start 2024-04-01T00:14:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aaf556640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aa806b7d0 0x7f6aa8071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aaf556640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6aa80724f0 0x7f6aa80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aaf556640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa80b0570 con 0x7f6aa815e120 2024-04-01T00:14:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aaf556640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa8072d10 con 0x7f6aa806b7d0 2024-04-01T00:14:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aaf556640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa80c6040 con 0x7f6aa80724f0 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aa806b7d0 0x7f6aa8071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aa806b7d0 0x7f6aa8071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58488/0 (socket says 172.21.15.64:58488) 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 -- 172.21.15.64:0/1905897791 learned_addr learned my addr 172.21.15.64:0/1905897791 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aadaf3640 1 --2- 172.21.15.64:0/1905897791 >> v2:172.21.15.100:3300/0 conn(0x7f6aa80724f0 0x7f6aa80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 -- 172.21.15.64:0/1905897791 >> v2:172.21.15.100:3300/0 conn(0x7f6aa80724f0 msgr2=0x7f6aa80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 --2- 172.21.15.64:0/1905897791 >> v2:172.21.15.100:3300/0 conn(0x7f6aa80724f0 0x7f6aa80cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 -- 172.21.15.64:0/1905897791 >> v1:172.21.15.100:6789/0 conn(0x7f6aa815e120 legacy=0x7f6aa806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.980+0000 7f6aacaf1640 1 -- 172.21.15.64:0/1905897791 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa80c6320 con 0x7f6aa806b7d0 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.981+0000 7f6aacaf1640 1 --2- 172.21.15.64:0/1905897791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8071fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80cb1f0 tx=0x7f6a9802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f4353e5bd7ccb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.981+0000 7f6a9ffff640 1 -- 172.21.15.64:0/1905897791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98034070 con 0x7f6aa806b7d0 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.981+0000 7f6a9ffff640 1 -- 172.21.15.64:0/1905897791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a98030860 con 0x7f6aa806b7d0 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.981+0000 7f6a9ffff640 1 -- 172.21.15.64:0/1905897791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98030c30 con 0x7f6aa806b7d0 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 -- 172.21.15.64:0/1905897791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 msgr2=0x7f6aa8071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 --2- 172.21.15.64:0/1905897791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8071fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80cb1f0 tx=0x7f6a9802b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 -- 172.21.15.64:0/1905897791 shutdown_connections 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 --2- 172.21.15.64:0/1905897791 >> v2:172.21.15.100:3300/0 conn(0x7f6aa80724f0 0x7f6aa80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 --2- 172.21.15.64:0/1905897791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8071fb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 -- 172.21.15.64:0/1905897791 >> 172.21.15.64:0/1905897791 conn(0x7f6aa80d0ba0 msgr2=0x7f6aa80d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 -- 172.21.15.64:0/1905897791 shutdown_connections 2024-04-01T00:14:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.982+0000 7f6aaf556640 1 -- 172.21.15.64:0/1905897791 wait complete. 2024-04-01T00:14:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.983+0000 7f6aaf556640 1 Processor -- start 2024-04-01T00:14:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.983+0000 7f6aaf556640 1 -- start start 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa80724f0 0x7f6aa80b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa815e120 0x7f6aa815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa815e500 con 0x7f6aa806b7d0 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa80c68e0 con 0x7f6aa80724f0 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa80c6e00 con 0x7f6aa815e120 2024-04-01T00:14:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58494/0 (socket says 172.21.15.64:58494) 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 -- 172.21.15.64:0/3996214934 learned_addr learned my addr 172.21.15.64:0/3996214934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aadaf3640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa815e120 0x7f6aa815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aacaf1640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa80724f0 0x7f6aa80b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 -- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa815e120 msgr2=0x7f6aa815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa815e120 0x7f6aa815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 -- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa80724f0 msgr2=0x7f6aa80b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa80724f0 0x7f6aa80b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 -- 172.21.15.64:0/3996214934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa8160180 con 0x7f6aa806b7d0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aad2f2640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400c3c0 tx=0x7f6aa400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40052d0 con 0x7f6aa806b7d0 2024-04-01T00:14:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4005490 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa80b67b0 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.984+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4007260 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.985+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa80b6a90 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.986+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa80681a0 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.986+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6aa4007400 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.986+0000 7f6a9dffb640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 0x7f6a8c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.987+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6aa4089920 con 0x7f6aa806b7d0 2024-04-01T00:14:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.987+0000 7f6aacaf1640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 0x7f6a8c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.987+0000 7f6aacaf1640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 0x7f6a8c06de50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a98036000 tx=0x7f6a98008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:42.991+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6aa4058230 con 0x7f6aa806b7d0 2024-04-01T00:14:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.173+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6aa80d19d0 con 0x7f6aa806b7d0 2024-04-01T00:14:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.173+0000 7f6a9dffb640 1 -- 172.21.15.64:0/3996214934 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f6aa405c2d0 con 0x7f6aa806b7d0 2024-04-01T00:14:43.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:43.174 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-04-01T00:14:43.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-04-01T00:14:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-04-01T00:14:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-04-01T00:14:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:43.175 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-04-01T00:14:43.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 msgr2=0x7f6a8c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 0x7f6a8c06de50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a98036000 tx=0x7f6a98008030 comp rx=0 tx=0).stop 2024-04-01T00:14:43.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 msgr2=0x7f6aa8160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400c3c0 tx=0x7f6aa400c880 comp rx=0 tx=0).stop 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 shutdown_connections 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6a8c06b990 0x7f6a8c06de50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa815e120 0x7f6aa815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa80724f0 0x7f6aa80b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 --2- 172.21.15.64:0/3996214934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa806b7d0 0x7f6aa8160bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 >> 172.21.15.64:0/3996214934 conn(0x7f6aa80d0ba0 msgr2=0x7f6aa815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.175+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 shutdown_connections 2024-04-01T00:14:43.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.176+0000 7f6aaf556640 1 -- 172.21.15.64:0/3996214934 wait complete. 2024-04-01T00:14:43.187 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-01T00:14:43.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-01T00:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.291+0000 7f033baeb640 1 Processor -- start 2024-04-01T00:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 -- start start 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03340715d0 0x7f0334071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f033406d570 0x7f033406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03340b3ca0 con 0x7f033415a430 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f03340b4260 con 0x7f033406d570 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.292+0000 7f033baeb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0334071140 con 0x7f03340715d0 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f033a088640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f033406d570 0x7f033406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f0339887640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f033415a430 0x7f03340b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33450/0 (socket says 172.21.15.64:33450) 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f0339887640 1 -- 172.21.15.64:0/3021292324 learned_addr learned my addr 172.21.15.64:0/3021292324 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f033a088640 1 -- 172.21.15.64:0/3021292324 >> v2:172.21.15.100:3300/0 conn(0x7f03340715d0 msgr2=0x7f0334071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f0339086640 1 --2- 172.21.15.64:0/3021292324 >> v2:172.21.15.100:3300/0 conn(0x7f03340715d0 0x7f0334071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f033a088640 1 --2- 172.21.15.64:0/3021292324 >> v2:172.21.15.100:3300/0 conn(0x7f03340715d0 0x7f0334071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f0338885640 1 -- 172.21.15.64:0/3021292324 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583074652 0 0) 0x7f03340b3ca0 con 0x7f033415a430 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f033a088640 1 -- 172.21.15.64:0/3021292324 >> v1:172.21.15.64:6789/0 conn(0x7f033415a430 legacy=0x7f03340b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.293+0000 7f033a088640 1 -- 172.21.15.64:0/3021292324 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03340b3f80 con 0x7f033406d570 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033a088640 1 --2- 172.21.15.64:0/3021292324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f033406d9e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f032c00bfa0 tx=0x7f032c02e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f03f355fcaac0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f0338885640 1 -- 172.21.15.64:0/3021292324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c02e9d0 con 0x7f033406d570 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f0338885640 1 -- 172.21.15.64:0/3021292324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c02eb70 con 0x7f033406d570 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f0338885640 1 -- 172.21.15.64:0/3021292324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c009540 con 0x7f033406d570 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 -- 172.21.15.64:0/3021292324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 msgr2=0x7f033406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 --2- 172.21.15.64:0/3021292324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f033406d9e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f032c00bfa0 tx=0x7f032c02e030 comp rx=0 tx=0).stop 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 -- 172.21.15.64:0/3021292324 shutdown_connections 2024-04-01T00:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 --2- 172.21.15.64:0/3021292324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f033406d9e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 --2- 172.21.15.64:0/3021292324 >> v2:172.21.15.100:3300/0 conn(0x7f03340715d0 0x7f0334071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 -- 172.21.15.64:0/3021292324 >> 172.21.15.64:0/3021292324 conn(0x7f03340b16c0 msgr2=0x7f03340b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 -- 172.21.15.64:0/3021292324 shutdown_connections 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.294+0000 7f033baeb640 1 -- 172.21.15.64:0/3021292324 wait complete. 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.295+0000 7f033baeb640 1 Processor -- start 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 -- start start 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03340715d0 0x7f0334152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033415a430 0x7f03341491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f033406cce0 con 0x7f033406d570 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03340b2b70 con 0x7f03340715d0 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f033baeb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f033406ce60 con 0x7f033415a430 2024-04-01T00:14:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f0339887640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f0339887640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58516/0 (socket says 172.21.15.64:58516) 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.296+0000 7f0339887640 1 -- 172.21.15.64:0/1357560713 learned_addr learned my addr 172.21.15.64:0/1357560713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339086640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03340715d0 0x7f0334152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f033a088640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033415a430 0x7f03341491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 -- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033415a430 msgr2=0x7f03341491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033415a430 0x7f03341491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 -- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03340715d0 msgr2=0x7f0334152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03340715d0 0x7f0334152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 -- 172.21.15.64:0/1357560713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033415bd10 con 0x7f033406d570 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f0339887640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f033000c430 tx=0x7f033000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300074e0 con 0x7f033406d570 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03300047e0 con 0x7f033406d570 2024-04-01T00:14:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033415ac70 con 0x7f033406d570 2024-04-01T00:14:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330004bd0 con 0x7f033406d570 2024-04-01T00:14:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.297+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033415b510 con 0x7f033406d570 2024-04-01T00:14:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.298+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03340681a0 con 0x7f033406d570 2024-04-01T00:14:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.299+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0330011070 con 0x7f033406d570 2024-04-01T00:14:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.299+0000 7f032a7fc640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 0x7f031806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.299+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f033008a0b0 con 0x7f033406d570 2024-04-01T00:14:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.299+0000 7f0339086640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 0x7f031806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.300+0000 7f0339086640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 0x7f031806de50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.303+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f03300590a0 con 0x7f033406d570 2024-04-01T00:14:43.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.484+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0334157870 con 0x7f033406d570 2024-04-01T00:14:43.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.552+0000 7f032a7fc640 1 -- 172.21.15.64:0/1357560713 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f03300589a0 con 0x7f033406d570 2024-04-01T00:14:43.553 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-04-01T00:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 msgr2=0x7f031806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 0x7f031806de50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324008040 comp rx=0 tx=0).stop 2024-04-01T00:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 msgr2=0x7f0334151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f033000c430 tx=0x7f033000c8f0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 shutdown_connections 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f031806b990 0x7f031806de50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033415a430 0x7f03341491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03340715d0 0x7f0334152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 --2- 172.21.15.64:0/1357560713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f033406d570 0x7f0334151f30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 >> 172.21.15.64:0/1357560713 conn(0x7f03340b16c0 msgr2=0x7f0334066e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 shutdown_connections 2024-04-01T00:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.554+0000 7f033baeb640 1 -- 172.21.15.64:0/1357560713 wait complete. 2024-04-01T00:14:43.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-01T00:14:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.676+0000 7f5048fbc640 1 Processor -- start 2024-04-01T00:14:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.676+0000 7f5048fbc640 1 -- start start 2024-04-01T00:14:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.677+0000 7f5048fbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 0x7f50440b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.677+0000 7f5048fbc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.677+0000 7f5048fbc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5044161e10 con 0x7f5044161a30 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.677+0000 7f5048fbc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f50440d12a0 con 0x7f50440d0900 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.677+0000 7f5048fbc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f50440cef20 con 0x7f50440d4a40 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5042d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5041d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 0x7f50440b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5042d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:34074/0 (socket says 172.21.15.64:34074) 2024-04-01T00:14:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5042d9d640 1 -- 172.21.15.64:0/2641786466 learned_addr learned my addr 172.21.15.64:0/2641786466 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5041d9b640 1 -- 172.21.15.64:0/2641786466 >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 msgr2=0x7f50440d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5041d9b640 1 --2- 172.21.15.64:0/2641786466 >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5041d9b640 1 -- 172.21.15.64:0/2641786466 >> v1:172.21.15.100:6789/0 conn(0x7f5044161a30 legacy=0x7f50440b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5041d9b640 1 -- 172.21.15.64:0/2641786466 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440d2880 con 0x7f50440d4a40 2024-04-01T00:14:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f504159a640 1 -- 172.21.15.64:0/2641786466 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970138231 0 0) 0x7f503c001110 con 0x7f5044161a30 2024-04-01T00:14:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.678+0000 7f5042d9d640 1 --2- 172.21.15.64:0/2641786466 >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.679+0000 7f5041d9b640 1 --2- 172.21.15.64:0/2641786466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 0x7f50440b3740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50380097a0 tx=0x7f503802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966bf2fd388e5b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.679+0000 7f504159a640 1 -- 172.21.15.64:0/2641786466 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038035020 con 0x7f50440d4a40 2024-04-01T00:14:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.679+0000 7f504159a640 1 -- 172.21.15.64:0/2641786466 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038031830 con 0x7f50440d4a40 2024-04-01T00:14:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.679+0000 7f504159a640 1 -- 172.21.15.64:0/2641786466 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038031c00 con 0x7f50440d4a40 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 -- 172.21.15.64:0/2641786466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 msgr2=0x7f50440b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 --2- 172.21.15.64:0/2641786466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 0x7f50440b3740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50380097a0 tx=0x7f503802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 -- 172.21.15.64:0/2641786466 shutdown_connections 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 --2- 172.21.15.64:0/2641786466 >> v2:172.21.15.100:3300/0 conn(0x7f50440d0900 0x7f50440d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 --2- 172.21.15.64:0/2641786466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50440d4a40 0x7f50440b3740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 -- 172.21.15.64:0/2641786466 >> 172.21.15.64:0/2641786466 conn(0x7f504409d250 msgr2=0x7f50440d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 -- 172.21.15.64:0/2641786466 shutdown_connections 2024-04-01T00:14:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.680+0000 7f5048fbc640 1 -- 172.21.15.64:0/2641786466 wait complete. 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 Processor -- start 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 -- start start 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 0x7f50440b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5044161a30 0x7f50440b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50440ceb40 con 0x7f50440d0900 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d2e00 con 0x7f50440d4a40 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.682+0000 7f5048fbc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f50440cf3b0 con 0x7f5044161a30 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59614/0 (socket says 172.21.15.64:59614) 2024-04-01T00:14:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 -- 172.21.15.64:0/1552035905 learned_addr learned my addr 172.21.15.64:0/1552035905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5041d9b640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 0x7f50440b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5042d9d640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5044161a30 0x7f50440b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 -- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5044161a30 msgr2=0x7f50440b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5044161a30 0x7f50440b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 -- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 msgr2=0x7f50440b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 0x7f50440b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 -- 172.21.15.64:0/1552035905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440cbd70 con 0x7f50440d0900 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5041d9b640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 0x7f50440b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f504259c640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c3c0 tx=0x7f503c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c017070 con 0x7f50440d0900 2024-04-01T00:14:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503c0035f0 con 0x7f50440d0900 2024-04-01T00:14:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50440cc050 con 0x7f50440d0900 2024-04-01T00:14:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0039e0 con 0x7f50440d0900 2024-04-01T00:14:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.683+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50440cc610 con 0x7f50440d0900 2024-04-01T00:14:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.684+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50440681a0 con 0x7f50440d0900 2024-04-01T00:14:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.685+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f503c01a070 con 0x7f50440d0900 2024-04-01T00:14:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.685+0000 7f5032ffd640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 0x7f502406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.686+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f503c0898c0 con 0x7f50440d0900 2024-04-01T00:14:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.686+0000 7f5041d9b640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 0x7f502406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.686+0000 7f5041d9b640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 0x7f502406de50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5038036000 tx=0x7f5038008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.690+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f5044161e10 con 0x7f50440d0900 2024-04-01T00:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:43.870+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f504406d5d0 con 0x7f50440d0900 2024-04-01T00:14:44.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.558+0000 7f5032ffd640 1 -- 172.21.15.64:0/1552035905 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f503c0580f0 con 0x7f50440d0900 2024-04-01T00:14:44.559 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-04-01T00:14:44.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.562+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 msgr2=0x7f502406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:44.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.562+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 0x7f502406de50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5038036000 tx=0x7f5038008030 comp rx=0 tx=0).stop 2024-04-01T00:14:44.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.562+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 msgr2=0x7f504415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:44.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.562+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c3c0 tx=0x7f503c00c880 comp rx=0 tx=0).stop 2024-04-01T00:14:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 shutdown_connections 2024-04-01T00:14:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f502406b990 0x7f502406de50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5044161a30 0x7f50440b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50440d4a40 0x7f50440b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 --2- 172.21.15.64:0/1552035905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50440d0900 0x7f504415bf90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 >> 172.21.15.64:0/1552035905 conn(0x7f504409d250 msgr2=0x7f504406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 shutdown_connections 2024-04-01T00:14:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.563+0000 7f5048fbc640 1 -- 172.21.15.64:0/1552035905 wait complete. 2024-04-01T00:14:44.587 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-01T00:14:44.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.695+0000 7f2cdbf25640 1 Processor -- start 2024-04-01T00:14:44.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.695+0000 7f2cdbf25640 1 -- start start 2024-04-01T00:14:44.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cdbf25640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd4161a30 0x7f2cd40d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cdbf25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cd40b3740 0x7f2cd40b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cdbf25640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cd4161e10 con 0x7f2cd40d0900 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cdbf25640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd40b3ba0 con 0x7f2cd4161a30 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cdbf25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd40cef20 con 0x7f2cd40b3740 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cd9cc1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd4161a30 0x7f2cd40d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cd9cc1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd4161a30 0x7f2cd40d4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59616/0 (socket says 172.21.15.64:59616) 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.696+0000 7f2cd9cc1640 1 -- 172.21.15.64:0/952746374 learned_addr learned my addr 172.21.15.64:0/952746374 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd94c0640 1 --2- 172.21.15.64:0/952746374 >> v2:172.21.15.100:3300/0 conn(0x7f2cd40b3740 0x7f2cd40b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd9cc1640 1 -- 172.21.15.64:0/952746374 >> v2:172.21.15.100:3300/0 conn(0x7f2cd40b3740 msgr2=0x7f2cd40b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd9cc1640 1 --2- 172.21.15.64:0/952746374 >> v2:172.21.15.100:3300/0 conn(0x7f2cd40b3740 0x7f2cd40b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd9cc1640 1 -- 172.21.15.64:0/952746374 >> v1:172.21.15.100:6789/0 conn(0x7f2cd40d0900 legacy=0x7f2cd40cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:44.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd9cc1640 1 -- 172.21.15.64:0/952746374 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd40d2880 con 0x7f2cd4161a30 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd9cc1640 1 --2- 172.21.15.64:0/952746374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd4161a30 0x7f2cd40d4a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64218f469e062cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.697+0000 7f2cd8cbf640 1 -- 172.21.15.64:0/952746374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc02ca80 con 0x7f2cd4161a30 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cd8cbf640 1 -- 172.21.15.64:0/952746374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02cc20 con 0x7f2cd4161a30 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cd8cbf640 1 -- 172.21.15.64:0/952746374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0343f0 con 0x7f2cd4161a30 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 -- 172.21.15.64:0/952746374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd4161a30 msgr2=0x7f2cd40d4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/952746374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd4161a30 0x7f2cd40d4a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02bee0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 -- 172.21.15.64:0/952746374 shutdown_connections 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/952746374 >> v2:172.21.15.100:3300/0 conn(0x7f2cd40b3740 0x7f2cd40b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/952746374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd4161a30 0x7f2cd40d4a40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 -- 172.21.15.64:0/952746374 >> 172.21.15.64:0/952746374 conn(0x7f2cd409d250 msgr2=0x7f2cd40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 -- 172.21.15.64:0/952746374 shutdown_connections 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.698+0000 7f2cdbf25640 1 -- 172.21.15.64:0/952746374 wait complete. 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.699+0000 7f2cdbf25640 1 Processor -- start 2024-04-01T00:14:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.699+0000 7f2cdbf25640 1 -- start start 2024-04-01T00:14:44.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd40b3740 0x7f2cd40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd4161a30 0x7f2cd40c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40d2e00 con 0x7f2cd40d0900 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40b5690 con 0x7f2cd4161a30 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cdbf25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd40ceb40 con 0x7f2cd40b3740 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59626/0 (socket says 172.21.15.64:59626) 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 -- 172.21.15.64:0/4082518258 learned_addr learned my addr 172.21.15.64:0/4082518258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd9cc1640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd40b3740 0x7f2cd40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cda4c2640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd4161a30 0x7f2cd40c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 -- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd40b3740 msgr2=0x7f2cd40ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd40b3740 0x7f2cd40ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 -- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd4161a30 msgr2=0x7f2cd40c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd4161a30 0x7f2cd40c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:44.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.700+0000 7f2cd94c0640 1 -- 172.21.15.64:0/4082518258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd40c7fe0 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cd94c0640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800d800 tx=0x7f2cc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800b6c0 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc8019070 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd40c8280 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800ba80 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.701+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd40c8840 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.702+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd40681a0 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.702+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2cc800bc20 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.703+0000 7f2cd27fc640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 0x7f2cc006de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.703+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f2cc808b4c0 con 0x7f2cd40d0900 2024-04-01T00:14:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.703+0000 7f2cd9cc1640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 0x7f2cc006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.704+0000 7f2cd9cc1640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 0x7f2cc006de80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc035000 tx=0x7f2ccc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:44.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.707+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2cc8059c50 con 0x7f2cd40d0900 2024-04-01T00:14:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:44.925+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2cd40cac40 con 0x7f2cd40d0900 2024-04-01T00:14:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.571+0000 7f2cd27fc640 1 -- 172.21.15.64:0/4082518258 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f2cc805dcf0 con 0x7f2cd40d0900 2024-04-01T00:14:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2024-04-01T00:14:45.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.576+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 msgr2=0x7f2cc006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.576+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 0x7f2cc006de80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc035000 tx=0x7f2ccc030000 comp rx=0 tx=0).stop 2024-04-01T00:14:45.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.576+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 msgr2=0x7f2cd40c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.576+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800d800 tx=0x7f2cc800dcc0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 shutdown_connections 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2cc006b9c0 0x7f2cc006de80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd4161a30 0x7f2cd40c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd40d0900 0x7f2cd40c9320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 --2- 172.21.15.64:0/4082518258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd40b3740 0x7f2cd40ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 >> 172.21.15.64:0/4082518258 conn(0x7f2cd409d250 msgr2=0x7f2cd415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 shutdown_connections 2024-04-01T00:14:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.579+0000 7f2cdbf25640 1 -- 172.21.15.64:0/4082518258 wait complete. 2024-04-01T00:14:45.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:45.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.698+0000 7fd50c5fd640 1 Processor -- start 2024-04-01T00:14:45.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.698+0000 7fd50c5fd640 1 -- start start 2024-04-01T00:14:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50c5fd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5040cec60 0x7fd5040d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50c5fd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5040d1650 con 0x7fd5040b7560 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50c5fd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5040cc320 con 0x7fd5040b7940 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50c5fd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040cc600 con 0x7fd5040cec60 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50a399640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd5040b7560 0x7fd5040ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41666/0 (socket says 172.21.15.64:41666) 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd50a399640 1 -- 172.21.15.64:0/2080376955 learned_addr learned my addr 172.21.15.64:0/2080376955 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.699+0000 7fd509b98640 1 --2- 172.21.15.64:0/2080376955 >> v2:172.21.15.64:3300/0 conn(0x7fd5040cec60 0x7fd5040d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.700+0000 7fd509b98640 1 -- 172.21.15.64:0/2080376955 >> v1:172.21.15.100:6789/0 conn(0x7fd5040b7940 legacy=0x7fd5040d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.700+0000 7fd509b98640 1 -- 172.21.15.64:0/2080376955 >> v1:172.21.15.64:6789/0 conn(0x7fd5040b7560 legacy=0x7fd5040ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.700+0000 7fd509b98640 1 -- 172.21.15.64:0/2080376955 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040cc8e0 con 0x7fd5040cec60 2024-04-01T00:14:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.700+0000 7fd509397640 1 -- 172.21.15.64:0/2080376955 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191950785 0 0) 0x7fd5040d1650 con 0x7fd5040b7560 2024-04-01T00:14:45.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.700+0000 7fd509b98640 1 --2- 172.21.15.64:0/2080376955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040d0040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80097a0 tx=0x7fd4f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9303e4527cf899b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd509397640 1 -- 172.21.15.64:0/2080376955 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8035020 con 0x7fd5040cec60 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd509397640 1 -- 172.21.15.64:0/2080376955 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f8031830 con 0x7fd5040cec60 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd509397640 1 -- 172.21.15.64:0/2080376955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8031c00 con 0x7fd5040cec60 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 -- 172.21.15.64:0/2080376955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 msgr2=0x7fd5040d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/2080376955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040d0040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80097a0 tx=0x7fd4f802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 -- 172.21.15.64:0/2080376955 shutdown_connections 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/2080376955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040d0040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 -- 172.21.15.64:0/2080376955 >> 172.21.15.64:0/2080376955 conn(0x7fd5040d1cd0 msgr2=0x7fd5040d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 -- 172.21.15.64:0/2080376955 shutdown_connections 2024-04-01T00:14:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.701+0000 7fd50c5fd640 1 -- 172.21.15.64:0/2080376955 wait complete. 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.702+0000 7fd50c5fd640 1 Processor -- start 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.702+0000 7fd50c5fd640 1 -- start start 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5040b7560 0x7fd5041f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5040b7940 0x7fd5040b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:45.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040b4b20 con 0x7fd5040cec60 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040ce0d0 con 0x7fd5040b7940 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50c5fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5040d2860 con 0x7fd5040b7560 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50ab9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50ab9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59648/0 (socket says 172.21.15.64:59648) 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50ab9a640 1 -- 172.21.15.64:0/1795865862 learned_addr learned my addr 172.21.15.64:0/1795865862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd50a399640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5040b7560 0x7fd5041f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.703+0000 7fd509b98640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5040b7940 0x7fd5040b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 -- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5040b7560 msgr2=0x7fd5041f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5040b7560 0x7fd5041f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 -- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5040b7940 msgr2=0x7fd5040b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5040b7940 0x7fd5040b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 -- 172.21.15.64:0/1795865862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5041f4500 con 0x7fd5040cec60 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50ab9a640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7b0 tx=0x7fd4f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4017070 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f4003da0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041f47e0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40052b0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.704+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041f4da0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.705+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5040cc0a0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.705+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd4f401a070 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.706+0000 7fd502ffd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 0x7fd4f006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.706+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd4f4089df0 con 0x7fd5040cec60 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.706+0000 7fd50a399640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 0x7fd4f006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.706+0000 7fd50a399640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 0x7fd4f006de50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc000f60 tx=0x7fd4fc007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.710+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd5040cc0a0 con 0x7fd5040cec60 2024-04-01T00:14:45.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.893+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5040c8ba0 con 0x7fd5040cec60 2024-04-01T00:14:45.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.894+0000 7fd502ffd640 1 -- 172.21.15.64:0/1795865862 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16658 (secure 0 0 0) 0x7fd5040c8ba0 con 0x7fd5040cec60 2024-04-01T00:14:45.895 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:45.895 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:44.926815+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:45.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 msgr2=0x7fd4f006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 0x7fd4f006de50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc000f60 tx=0x7fd4fc007d50 comp rx=0 tx=0).stop 2024-04-01T00:14:45.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 msgr2=0x7fd5040b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7b0 tx=0x7fd4f400cc70 comp rx=0 tx=0).stop 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 shutdown_connections 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4f006b990 0x7fd4f006de50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5040cec60 0x7fd5040b51e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5040b7940 0x7fd5040b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 --2- 172.21.15.64:0/1795865862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5040b7560 0x7fd5041f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 >> 172.21.15.64:0/1795865862 conn(0x7fd5040d1cd0 msgr2=0x7fd5040b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 shutdown_connections 2024-04-01T00:14:45.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:45.896+0000 7fd50c5fd640 1 -- 172.21.15.64:0/1795865862 wait complete. 2024-04-01T00:14:45.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:46.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.009+0000 7f79a7d6c640 1 Processor -- start 2024-04-01T00:14:46.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.009+0000 7f79a7d6c640 1 -- start start 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a7d6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a7d6c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79a0072af0 0x7f79a00d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a7d6c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a0073150 con 0x7f79a00d0190 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a7d6c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f79a0073430 con 0x7f79a0072af0 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a7d6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f79a006d790 con 0x7f79a0158680 2024-04-01T00:14:46.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79a0072af0 0x7f79a00d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79a0072af0 0x7f79a00d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59652/0 (socket says 172.21.15.64:59652) 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 -- 172.21.15.64:0/4077106560 learned_addr learned my addr 172.21.15.64:0/4077106560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a5b08640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 -- 172.21.15.64:0/4077106560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 msgr2=0x7f79a00cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a00cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 -- 172.21.15.64:0/4077106560 >> v1:172.21.15.100:6789/0 conn(0x7f79a00d0190 legacy=0x7f79a00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.010+0000 7f79a6309640 1 -- 172.21.15.64:0/4077106560 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a006da70 con 0x7f79a0072af0 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.011+0000 7f79a6309640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a00d0610 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7998009e50 tx=0x7f799802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bad24be19dc85cc server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.011+0000 7f79a4b06640 1 -- 172.21.15.64:0/4077106560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799802ca70 con 0x7f79a0072af0 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.011+0000 7f79a4b06640 1 -- 172.21.15.64:0/4077106560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799802cc10 con 0x7f79a0072af0 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.011+0000 7f79a4b06640 1 -- 172.21.15.64:0/4077106560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980343f0 con 0x7f79a0072af0 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 -- 172.21.15.64:0/4077106560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 msgr2=0x7f79a00d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a00d0610 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7998009e50 tx=0x7f799802bed0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 -- 172.21.15.64:0/4077106560 shutdown_connections 2024-04-01T00:14:46.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a00d0610 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/4077106560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a00cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 -- 172.21.15.64:0/4077106560 >> 172.21.15.64:0/4077106560 conn(0x7f79a006b2a0 msgr2=0x7f79a006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:46.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 -- 172.21.15.64:0/4077106560 shutdown_connections 2024-04-01T00:14:46.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.012+0000 7f79a7d6c640 1 -- 172.21.15.64:0/4077106560 wait complete. 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.013+0000 7f79a7d6c640 1 Processor -- start 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.013+0000 7f79a7d6c640 1 -- start start 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a00d0190 0x7f79a0161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a009ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79a006e270 con 0x7f79a0072af0 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0158a60 con 0x7f79a00d0190 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f79a006e3f0 con 0x7f79a0158680 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59666/0 (socket says 172.21.15.64:59666) 2024-04-01T00:14:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 -- 172.21.15.64:0/3919199985 learned_addr learned my addr 172.21.15.64:0/3919199985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 -- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 msgr2=0x7f79a009ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a009ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 -- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a00d0190 msgr2=0x7f79a0161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a00d0190 0x7f79a0161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 -- 172.21.15.64:0/3919199985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a01f3bb0 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a5b08640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d5d0 tx=0x7f799c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0125f0 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c019070 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a01f3e90 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0129f0 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.014+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01f4450 con 0x7f79a0072af0 2024-04-01T00:14:46.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.015+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a00681a0 con 0x7f79a0072af0 2024-04-01T00:14:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.016+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f799c012b90 con 0x7f79a0072af0 2024-04-01T00:14:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.016+0000 7f79967fc640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 0x7f798406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.016+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f799c08ac90 con 0x7f79a0072af0 2024-04-01T00:14:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.017+0000 7f79a5307640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 0x7f798406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.017+0000 7f79a5307640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 0x7f798406de50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7990000f60 tx=0x7f79900079d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.021+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f799c0598a0 con 0x7f79a0072af0 2024-04-01T00:14:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.201+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79a00cf150 con 0x7f79a0072af0 2024-04-01T00:14:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.202+0000 7f79967fc640 1 -- 172.21.15.64:0/3919199985 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16658 (secure 0 0 0) 0x7f799c0591a0 con 0x7f79a0072af0 2024-04-01T00:14:46.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:46.203 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:44.926815+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 msgr2=0x7f798406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 0x7f798406de50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7990000f60 tx=0x7f79900079d0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 msgr2=0x7f79a0143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d5d0 tx=0x7f799c00da90 comp rx=0 tx=0).stop 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 shutdown_connections 2024-04-01T00:14:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f798406b990 0x7f798406de50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a0158680 0x7f79a009ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a00d0190 0x7f79a0161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 --2- 172.21.15.64:0/3919199985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a0072af0 0x7f79a0143d30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 >> 172.21.15.64:0/3919199985 conn(0x7f79a006b2a0 msgr2=0x7f79a015f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 shutdown_connections 2024-04-01T00:14:46.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.204+0000 7f79a7d6c640 1 -- 172.21.15.64:0/3919199985 wait complete. 2024-04-01T00:14:46.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-01T00:14:46.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 Processor -- start 2024-04-01T00:14:46.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 -- start start 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f40cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25f406cee0 0x7f25f415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f40b1ce0 con 0x7f25f415de20 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40cf4d0 con 0x7f25f406cee0 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.321+0000 7f25fa4c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40b1310 con 0x7f25f406cb00 2024-04-01T00:14:46.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f8a5d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f25f415de20 0x7f25f4161c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41676/0 (socket says 172.21.15.64:41676) 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f8a5d640 1 -- 172.21.15.64:0/1432348716 learned_addr learned my addr 172.21.15.64:0/1432348716 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f37fe640 1 --2- 172.21.15.64:0/1432348716 >> v2:172.21.15.100:3300/0 conn(0x7f25f406cee0 0x7f25f415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f3fff640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f40cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385378487 0 0) 0x7f25f40b1ce0 con 0x7f25f415de20 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25dc003520 con 0x7f25f415de20 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2318833725 0 0) 0x7f25dc003520 con 0x7f25f415de20 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f37fe640 1 -- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 msgr2=0x7f25f40cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f37fe640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f40cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f37fe640 1 -- 172.21.15.64:0/1432348716 >> v1:172.21.15.64:6789/0 conn(0x7f25f415de20 legacy=0x7f25f4161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f37fe640 1 -- 172.21.15.64:0/1432348716 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40b1fa0 con 0x7f25f406cee0 2024-04-01T00:14:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.322+0000 7f25f3fff640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f40cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.323+0000 7f25f37fe640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f406cee0 0x7f25f415d8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25e80097a0 tx=0x7f25e802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e8947c0d3812ac server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.323+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8037070 con 0x7f25f406cee0 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.323+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8030510 con 0x7f25f406cee0 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.323+0000 7f25f2ffd640 1 -- 172.21.15.64:0/1432348716 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec0019f0 con 0x7f25f406cee0 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 -- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f406cee0 msgr2=0x7f25f415d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f406cee0 0x7f25f415d8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25e80097a0 tx=0x7f25e802b790 comp rx=0 tx=0).stop 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 -- 172.21.15.64:0/1432348716 shutdown_connections 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f406cee0 0x7f25f415d8e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/1432348716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f40cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 -- 172.21.15.64:0/1432348716 >> 172.21.15.64:0/1432348716 conn(0x7f25f409a980 msgr2=0x7f25f409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 -- 172.21.15.64:0/1432348716 shutdown_connections 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.324+0000 7f25fa4c0640 1 -- 172.21.15.64:0/1432348716 wait complete. 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.325+0000 7f25fa4c0640 1 Processor -- start 2024-04-01T00:14:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.325+0000 7f25fa4c0640 1 -- start start 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f415de20 0x7f25f40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40b1960 con 0x7f25f406cee0 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40b2560 con 0x7f25f415de20 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f25f415e2b0 con 0x7f25f406cb00 2024-04-01T00:14:46.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59674/0 (socket says 172.21.15.64:59674) 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 -- 172.21.15.64:0/3301995568 learned_addr learned my addr 172.21.15.64:0/3301995568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f8a5d640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f415de20 0x7f25f40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f3fff640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 -- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 msgr2=0x7f25f414b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f414b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 -- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f415de20 msgr2=0x7f25f40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f415de20 0x7f25f40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 -- 172.21.15.64:0/3301995568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f415e7a0 con 0x7f25f406cee0 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f37fe640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8041020 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e803c870 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f415ea40 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.326+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8037040 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.327+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f414dfc0 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.328+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f414e2b0 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.328+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f25e8044020 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.328+0000 7f25f0ff9640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 0x7f25e006de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.328+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f25e80abd50 con 0x7f25f406cee0 2024-04-01T00:14:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.328+0000 7f25f3fff640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 0x7f25e006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.329+0000 7f25f3fff640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 0x7f25e006de80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009560 tx=0x7f25e4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.333+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f25e80af030 con 0x7f25f406cee0 2024-04-01T00:14:46.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:14:46.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:14:46.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.553+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f25f40681a0 con 0x7f25f406cee0 2024-04-01T00:14:46.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.577+0000 7f25f0ff9640 1 -- 172.21.15.64:0/3301995568 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f25e804b070 con 0x7f25f406cee0 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 msgr2=0x7f25e006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 0x7f25e006de80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009560 tx=0x7f25e4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 msgr2=0x7f25f40cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e80046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 shutdown_connections 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f25e006b9c0 0x7f25e006de80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f415de20 0x7f25f40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25f406cee0 0x7f25f40cfdc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 --2- 172.21.15.64:0/3301995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25f406cb00 0x7f25f414b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 >> 172.21.15.64:0/3301995568 conn(0x7f25f409a980 msgr2=0x7f25f40af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:46.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 shutdown_connections 2024-04-01T00:14:46.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.579+0000 7f25fa4c0640 1 -- 172.21.15.64:0/3301995568 wait complete. 2024-04-01T00:14:46.590 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-01T00:14:46.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-01T00:14:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 Processor -- start 2024-04-01T00:14:46.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 -- start start 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fe40b3740 0x7f8fe40b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe4161e10 con 0x7f8fe40d0900 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe40b3ba0 con 0x7f8fe40b3740 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.735+0000 7f8fea82a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe40cef20 con 0x7f8fe4161a30 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fe40b3740 0x7f8fe40b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe8dc7640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8fe40d0900 0x7f8fe40cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41680/0 (socket says 172.21.15.64:41680) 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fe40b3740 0x7f8fe40b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59680/0 (socket says 172.21.15.64:59680) 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe8dc7640 1 -- 172.21.15.64:0/1785468309 learned_addr learned my addr 172.21.15.64:0/1785468309 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe3fff640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 -- 172.21.15.64:0/1785468309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 msgr2=0x7f8fe40d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 -- 172.21.15.64:0/1785468309 >> v1:172.21.15.64:6789/0 conn(0x7f8fe40d0900 legacy=0x7f8fe40cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe2ffd640 1 -- 172.21.15.64:0/1785468309 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616161113 0 0) 0x7f8fdc001490 con 0x7f8fe40d0900 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.736+0000 7f8fe37fe640 1 -- 172.21.15.64:0/1785468309 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40d2880 con 0x7f8fe40b3740 2024-04-01T00:14:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fe37fe640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40b3740 0x7f8fe40b5150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097a0 tx=0x7f8fd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d151ba2353f8a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fe2ffd640 1 -- 172.21.15.64:0/1785468309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8035020 con 0x7f8fe40b3740 2024-04-01T00:14:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fe2ffd640 1 -- 172.21.15.64:0/1785468309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd8031830 con 0x7f8fe40b3740 2024-04-01T00:14:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fe2ffd640 1 -- 172.21.15.64:0/1785468309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8031c00 con 0x7f8fe40b3740 2024-04-01T00:14:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 -- 172.21.15.64:0/1785468309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40b3740 msgr2=0x7f8fe40b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40b3740 0x7f8fe40b5150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097a0 tx=0x7f8fd802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 -- 172.21.15.64:0/1785468309 shutdown_connections 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40b3740 0x7f8fe40b5150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 --2- 172.21.15.64:0/1785468309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 -- 172.21.15.64:0/1785468309 >> 172.21.15.64:0/1785468309 conn(0x7f8fe409d250 msgr2=0x7f8fe40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 -- 172.21.15.64:0/1785468309 shutdown_connections 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.737+0000 7f8fea82a640 1 -- 172.21.15.64:0/1785468309 wait complete. 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.738+0000 7f8fea82a640 1 Processor -- start 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 -- start start 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fe40b3740 0x7f8fe415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40d2e00 con 0x7f8fe40d0900 2024-04-01T00:14:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40b5690 con 0x7f8fe40b3740 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fea82a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe40ceb40 con 0x7f8fe4161a30 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fe37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fe37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59684/0 (socket says 172.21.15.64:59684) 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fe37fe640 1 -- 172.21.15.64:0/2293720761 learned_addr learned my addr 172.21.15.64:0/2293720761 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.739+0000 7f8fe3fff640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fe40b3740 0x7f8fe415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe8dc7640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 -- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 msgr2=0x7f8fe40b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 -- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fe40b3740 msgr2=0x7f8fe415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fe40b3740 0x7f8fe415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 -- 172.21.15.64:0/2293720761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe415aa20 con 0x7f8fe40d0900 2024-04-01T00:14:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe37fe640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8034040 tx=0x7f8fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8040070 con 0x7f8fe40d0900 2024-04-01T00:14:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd803a850 con 0x7f8fe40d0900 2024-04-01T00:14:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe40c9240 con 0x7f8fe40d0900 2024-04-01T00:14:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8035040 con 0x7f8fe40d0900 2024-04-01T00:14:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.740+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe415e770 con 0x7f8fe40d0900 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.741+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe406d8e0 con 0x7f8fe40d0900 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.742+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f8fd8038020 con 0x7f8fe40d0900 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.742+0000 7f8fe0ff9640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 0x7f8fd006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.742+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8fd80abee0 con 0x7f8fe40d0900 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.742+0000 7f8fe3fff640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 0x7f8fd006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.743+0000 7f8fe3fff640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 0x7f8fd006de50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009b40 tx=0x7f8fd4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.747+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f8fe406d8e0 con 0x7f8fe40d0900 2024-04-01T00:14:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:46.967+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8fe40d2660 con 0x7f8fe40d0900 2024-04-01T00:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.580+0000 7f8fe0ff9640 1 -- 172.21.15.64:0/2293720761 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f8fe40d2660 con 0x7f8fe40d0900 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.582+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 msgr2=0x7f8fd006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.582+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 0x7f8fd006de50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009b40 tx=0x7f8fd4009190 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.582+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 msgr2=0x7f8fe4160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.582+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8034040 tx=0x7f8fd80046c0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 shutdown_connections 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f8fd006b990 0x7f8fd006de50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fe4161a30 0x7f8fe40b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe40d0900 0x7f8fe4160490 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 --2- 172.21.15.64:0/2293720761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fe40b3740 0x7f8fe415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 >> 172.21.15.64:0/2293720761 conn(0x7f8fe409d250 msgr2=0x7f8fe40b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:47.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 shutdown_connections 2024-04-01T00:14:47.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.583+0000 7f8fea82a640 1 -- 172.21.15.64:0/2293720761 wait complete. 2024-04-01T00:14:47.596 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-01T00:14:47.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 Processor -- start 2024-04-01T00:14:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 -- start start 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33ac151fe0 0x7f33ac15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f33ac0cff40 0x7f33ac0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac071460 con 0x7f33ac15f610 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.700+0000 7f33b1355640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f33ac0d08e0 con 0x7f33ac151fe0 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33b1355640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f33ac15f180 con 0x7f33ac0cff40 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33ac151fe0 0x7f33ac15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33ac151fe0 0x7f33ac15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59694/0 (socket says 172.21.15.64:59694) 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aa7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f33ac15f610 0x7f33ac070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41692/0 (socket says 172.21.15.64:41692) 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 -- 172.21.15.64:0/3008546161 learned_addr learned my addr 172.21.15.64:0/3008546161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33ab7fe640 1 --2- 172.21.15.64:0/3008546161 >> v2:172.21.15.100:3300/0 conn(0x7f33ac0cff40 0x7f33ac0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 -- 172.21.15.64:0/3008546161 >> v2:172.21.15.100:3300/0 conn(0x7f33ac0cff40 msgr2=0x7f33ac0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 --2- 172.21.15.64:0/3008546161 >> v2:172.21.15.100:3300/0 conn(0x7f33ac0cff40 0x7f33ac0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 -- 172.21.15.64:0/3008546161 >> v1:172.21.15.64:6789/0 conn(0x7f33ac15f610 legacy=0x7f33ac070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33a9ffb640 1 -- 172.21.15.64:0/3008546161 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749421380 0 0) 0x7f33ac071460 con 0x7f33ac15f610 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.701+0000 7f33aaffd640 1 -- 172.21.15.64:0/3008546161 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac071740 con 0x7f33ac151fe0 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33aaffd640 1 --2- 172.21.15.64:0/3008546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac15beb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33ac06f180 tx=0x7f339c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf9e0cff446d111 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33a9ffb640 1 -- 172.21.15.64:0/3008546161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c034070 con 0x7f33ac151fe0 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33a9ffb640 1 -- 172.21.15.64:0/3008546161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c030860 con 0x7f33ac151fe0 2024-04-01T00:14:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33a9ffb640 1 -- 172.21.15.64:0/3008546161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c030c30 con 0x7f33ac151fe0 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 -- 172.21.15.64:0/3008546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 msgr2=0x7f33ac15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 --2- 172.21.15.64:0/3008546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac15beb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33ac06f180 tx=0x7f339c02b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 -- 172.21.15.64:0/3008546161 shutdown_connections 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 --2- 172.21.15.64:0/3008546161 >> v2:172.21.15.100:3300/0 conn(0x7f33ac0cff40 0x7f33ac0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 --2- 172.21.15.64:0/3008546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac15beb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 -- 172.21.15.64:0/3008546161 >> 172.21.15.64:0/3008546161 conn(0x7f33ac06b4e0 msgr2=0x7f33ac06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 -- 172.21.15.64:0/3008546161 shutdown_connections 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.702+0000 7f33b1355640 1 -- 172.21.15.64:0/3008546161 wait complete. 2024-04-01T00:14:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 Processor -- start 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 -- start start 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33ac0cff40 0x7f33ac1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33ac15f610 0x7f33ac14b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac164af0 con 0x7f33ac151fe0 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac15ee10 con 0x7f33ac15f610 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33b1355640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac164c70 con 0x7f33ac0cff40 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.704+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59700/0 (socket says 172.21.15.64:59700) 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 -- 172.21.15.64:0/2263918805 learned_addr learned my addr 172.21.15.64:0/2263918805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aaffd640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33ac0cff40 0x7f33ac1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33ab7fe640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33ac15f610 0x7f33ac14b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 -- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33ac0cff40 msgr2=0x7f33ac1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33ac0cff40 0x7f33ac1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 -- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33ac15f610 msgr2=0x7f33ac14b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33ac15f610 0x7f33ac14b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 -- 172.21.15.64:0/2263918805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac148020 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33aa7fc640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c410 tx=0x7f33a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00074e0 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a00047e0 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac148300 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004bd0 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.705+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac1488c0 con 0x7f33ac151fe0 2024-04-01T00:14:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.706+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac0681a0 con 0x7f33ac151fe0 2024-04-01T00:14:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.707+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f33a0011070 con 0x7f33ac151fe0 2024-04-01T00:14:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.707+0000 7f33937fe640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 0x7f339806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.707+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f33a008a350 con 0x7f33ac151fe0 2024-04-01T00:14:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.707+0000 7f33aaffd640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 0x7f339806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.708+0000 7f33aaffd640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 0x7f339806de80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.711+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f33a0058910 con 0x7f33ac151fe0 2024-04-01T00:14:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.928+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33ac0cdf10 con 0x7f33ac151fe0 2024-04-01T00:14:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.929+0000 7f33937fe640 1 -- 172.21.15.64:0/2263918805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7f33a005c9b0 con 0x7f33ac151fe0 2024-04-01T00:14:47.930 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:47.930 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T00:14:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T00:14:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 msgr2=0x7f339806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 0x7f339806de80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c008030 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 msgr2=0x7f33ac0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c410 tx=0x7f33a000c8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 shutdown_connections 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f339806b9c0 0x7f339806de80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33ac15f610 0x7f33ac14b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33ac151fe0 0x7f33ac0aec70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 --2- 172.21.15.64:0/2263918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33ac0cff40 0x7f33ac1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 >> 172.21.15.64:0/2263918805 conn(0x7f33ac06b4e0 msgr2=0x7f33ac0ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 shutdown_connections 2024-04-01T00:14:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:47.930+0000 7f33b1355640 1 -- 172.21.15.64:0/2263918805 wait complete. 2024-04-01T00:14:47.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T00:14:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.043+0000 7fbcf93e5640 1 Processor -- start 2024-04-01T00:14:48.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.043+0000 7fbcf93e5640 1 -- start start 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbcf41619e0 0x7fbcf40d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf40b3820 0x7fbcf40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbcf40cf910 0x7fbcf40cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf40b3c80 con 0x7fbcf40cf910 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf40d2ba0 con 0x7fbcf41619e0 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.044+0000 7fbcf93e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf40d2e80 con 0x7fbcf40b3820 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbcf40cf910 0x7fbcf40cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbcf40cf910 0x7fbcf40cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59708/0 (socket says 172.21.15.64:59708) 2024-04-01T00:14:48.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 -- 172.21.15.64:0/2262186928 learned_addr learned my addr 172.21.15.64:0/2262186928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf27fc640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf40b3820 0x7fbcf40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf2ffd640 1 --2- 172.21.15.64:0/2262186928 >> v2:172.21.15.100:3300/0 conn(0x7fbcf41619e0 0x7fbcf40d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 -- 172.21.15.64:0/2262186928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf40b3820 msgr2=0x7fbcf40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf40b3820 0x7fbcf40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 -- 172.21.15.64:0/2262186928 >> v2:172.21.15.100:3300/0 conn(0x7fbcf41619e0 msgr2=0x7fbcf40d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 --2- 172.21.15.64:0/2262186928 >> v2:172.21.15.100:3300/0 conn(0x7fbcf41619e0 0x7fbcf40d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.045+0000 7fbcf37fe640 1 -- 172.21.15.64:0/2262186928 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf40ceeb0 con 0x7fbcf40cf910 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf37fe640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf40cfda0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b6b40 tx=0x7fbcec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20820e57f2ef906f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf1ffb640 1 -- 172.21.15.64:0/2262186928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec02e070 con 0x7fbcf40cf910 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf1ffb640 1 -- 172.21.15.64:0/2262186928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec037070 con 0x7fbcf40cf910 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf1ffb640 1 -- 172.21.15.64:0/2262186928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec031a00 con 0x7fbcf40cf910 2024-04-01T00:14:48.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 -- 172.21.15.64:0/2262186928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 msgr2=0x7fbcf40cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf40cfda0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b6b40 tx=0x7fbcec02b880 comp rx=0 tx=0).stop 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 -- 172.21.15.64:0/2262186928 shutdown_connections 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf40cfda0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/2262186928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf40b3820 0x7fbcf40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/2262186928 >> v2:172.21.15.100:3300/0 conn(0x7fbcf41619e0 0x7fbcf40d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 -- 172.21.15.64:0/2262186928 >> 172.21.15.64:0/2262186928 conn(0x7fbcf409d290 msgr2=0x7fbcf40d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 -- 172.21.15.64:0/2262186928 shutdown_connections 2024-04-01T00:14:48.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.046+0000 7fbcf93e5640 1 -- 172.21.15.64:0/2262186928 wait complete. 2024-04-01T00:14:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 Processor -- start 2024-04-01T00:14:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 -- start start 2024-04-01T00:14:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf40b3820 0x7fbcf40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf41619e0 0x7fbcf40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf40b31b0 con 0x7fbcf40cf910 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf40d2890 con 0x7fbcf40b3820 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.048+0000 7fbcf93e5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf40ceb10 con 0x7fbcf41619e0 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59710/0 (socket says 172.21.15.64:59710) 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 -- 172.21.15.64:0/183871393 learned_addr learned my addr 172.21.15.64:0/183871393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf2ffd640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf40b3820 0x7fbcf40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf37fe640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf41619e0 0x7fbcf40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 -- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf41619e0 msgr2=0x7fbcf40cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf41619e0 0x7fbcf40cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 -- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf40b3820 msgr2=0x7fbcf40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf40b3820 0x7fbcf40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 -- 172.21.15.64:0/183871393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf40ca950 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf27fc640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e890 tx=0x7fbcdc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc017070 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc00c940 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf4160600 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00cd10 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.049+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf40c8570 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.050+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf40c9020 con 0x7fbcf40cf910 2024-04-01T00:14:48.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.051+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fbcdc01a070 con 0x7fbcf40cf910 2024-04-01T00:14:48.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.051+0000 7fbceb7fe640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 0x7fbcd406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.051+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbcdc08a7b0 con 0x7fbcf40cf910 2024-04-01T00:14:48.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.051+0000 7fbcf2ffd640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 0x7fbcd406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.052+0000 7fbcf2ffd640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 0x7fbcd406de80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011340 tx=0x7fbce400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.055+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fbcf40ca950 con 0x7fbcf40cf910 2024-04-01T00:14:48.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.238+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf40b5730 con 0x7fbcf40cf910 2024-04-01T00:14:48.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.239+0000 7fbceb7fe640 1 -- 172.21.15.64:0/183871393 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16645 (secure 0 0 0) 0x7fbcdc058d70 con 0x7fbcf40cf910 2024-04-01T00:14:48.240 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:48.240 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:46.560584+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 msgr2=0x7fbcd406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 0x7fbcd406de80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011340 tx=0x7fbce400b040 comp rx=0 tx=0).stop 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 msgr2=0x7fbcf415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e890 tx=0x7fbcdc00ed50 comp rx=0 tx=0).stop 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 shutdown_connections 2024-04-01T00:14:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbcd406b9c0 0x7fbcd406de80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcf41619e0 0x7fbcf40cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcf40cf910 0x7fbcf415c7f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 --2- 172.21.15.64:0/183871393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf40b3820 0x7fbcf40ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 >> 172.21.15.64:0/183871393 conn(0x7fbcf409d290 msgr2=0x7fbcf406c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 shutdown_connections 2024-04-01T00:14:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.241+0000 7fbcf93e5640 1 -- 172.21.15.64:0/183871393 wait complete. 2024-04-01T00:14:48.256 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-01T00:14:48.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:14:48.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.358+0000 7f28a34ed640 1 Processor -- start 2024-04-01T00:14:48.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 -- start start 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f289c151fe0 0x7f289c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c0cff40 0x7f289c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f289c071460 con 0x7f289c15f610 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f289c0d08e0 con 0x7f289c0cff40 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.359+0000 7f28a34ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f289c15f180 con 0x7f289c151fe0 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c0cff40 0x7f289c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c0cff40 0x7f289c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59712/0 (socket says 172.21.15.64:59712) 2024-04-01T00:14:48.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 -- 172.21.15.64:0/357711883 learned_addr learned my addr 172.21.15.64:0/357711883 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1289640 1 --2- 172.21.15.64:0/357711883 >> v2:172.21.15.100:3300/0 conn(0x7f289c151fe0 0x7f289c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 -- 172.21.15.64:0/357711883 >> v2:172.21.15.100:3300/0 conn(0x7f289c151fe0 msgr2=0x7f289c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 --2- 172.21.15.64:0/357711883 >> v2:172.21.15.100:3300/0 conn(0x7f289c151fe0 0x7f289c15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 -- 172.21.15.64:0/357711883 >> v1:172.21.15.100:6789/0 conn(0x7f289c15f610 legacy=0x7f289c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.360+0000 7f28a1a8a640 1 -- 172.21.15.64:0/357711883 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c071740 con 0x7f289c0cff40 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a1a8a640 1 --2- 172.21.15.64:0/357711883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c0d03a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f289c0cf440 tx=0x7f288c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4e26e3e2815b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f289bfff640 1 -- 172.21.15.64:0/357711883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c02cb20 con 0x7f289c0cff40 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f289bfff640 1 -- 172.21.15.64:0/357711883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288c02ccc0 con 0x7f289c0cff40 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f289bfff640 1 -- 172.21.15.64:0/357711883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c034610 con 0x7f289c0cff40 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 -- 172.21.15.64:0/357711883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 msgr2=0x7f289c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 --2- 172.21.15.64:0/357711883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c0d03a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f289c0cf440 tx=0x7f288c02bf80 comp rx=0 tx=0).stop 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 -- 172.21.15.64:0/357711883 shutdown_connections 2024-04-01T00:14:48.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 --2- 172.21.15.64:0/357711883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c0d03a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 --2- 172.21.15.64:0/357711883 >> v2:172.21.15.100:3300/0 conn(0x7f289c151fe0 0x7f289c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 -- 172.21.15.64:0/357711883 >> 172.21.15.64:0/357711883 conn(0x7f289c06b4e0 msgr2=0x7f289c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 -- 172.21.15.64:0/357711883 shutdown_connections 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.361+0000 7f28a34ed640 1 -- 172.21.15.64:0/357711883 wait complete. 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.362+0000 7f28a34ed640 1 Processor -- start 2024-04-01T00:14:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 -- start start 2024-04-01T00:14:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f289c151fe0 0x7f289c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 0x7f289c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f289c164af0 con 0x7f289c0cff40 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f289c15ee10 con 0x7f289c151fe0 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a34ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f289c164c70 con 0x7f289c15f610 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a1289640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a1289640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59726/0 (socket says 172.21.15.64:59726) 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a1289640 1 -- 172.21.15.64:0/1933830420 learned_addr learned my addr 172.21.15.64:0/1933830420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a1a8a640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 0x7f289c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.363+0000 7f28a0a88640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f289c151fe0 0x7f289c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 -- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 msgr2=0x7f289c14c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 0x7f289c14c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 -- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f289c151fe0 msgr2=0x7f289c14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f289c151fe0 0x7f289c14be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 -- 172.21.15.64:0/1933830420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c0ad650 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1a8a640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 0x7f289c14c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a1289640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f289400e9b0 tx=0x7f289400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894017070 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289400c9a0 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c0ad110 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289400cd90 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.364+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c0adbd0 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.365+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c0681a0 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.365+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f289401a070 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.366+0000 7f2899ffb640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 0x7f288806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.366+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f289408ac30 con 0x7f289c0cff40 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.366+0000 7f28a0a88640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 0x7f288806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.366+0000 7f28a0a88640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 0x7f288806de50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f289c15b130 tx=0x7f2890009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.370+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2894059840 con 0x7f289c0cff40 2024-04-01T00:14:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.547+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f289c0b0030 con 0x7f288806b990 2024-04-01T00:14:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.559+0000 7f2899ffb640 1 -- 172.21.15.64:0/1933830420 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f289c0b0030 con 0x7f288806b990 2024-04-01T00:14:48.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 msgr2=0x7f288806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 0x7f288806de50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f289c15b130 tx=0x7f2890009040 comp rx=0 tx=0).stop 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 msgr2=0x7f289c145f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f289400e9b0 tx=0x7f289400ee70 comp rx=0 tx=0).stop 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 shutdown_connections 2024-04-01T00:14:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f288806b990 0x7f288806de50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f289c15f610 0x7f289c14c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f289c151fe0 0x7f289c14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 --2- 172.21.15.64:0/1933830420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cff40 0x7f289c145f20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 >> 172.21.15.64:0/1933830420 conn(0x7f289c06b4e0 msgr2=0x7f289c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 shutdown_connections 2024-04-01T00:14:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.562+0000 7f28a34ed640 1 -- 172.21.15.64:0/1933830420 wait complete. 2024-04-01T00:14:48.574 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-04-01T00:14:46.755758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219968,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754752,"statfs":{"total":773094113280,"available":772868866048,"internally_reserved":0,"allocated":3899392,"data_stored":2211398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000811"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567529+0000","last_change":"2024-04-01T00:14:44.567529+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567529+0000","last_undegraded":"2024-04-01T00:14:44.567529+0000","last_fullsized":"2024-04-01T00:14:44.567529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568712+0000","last_change":"2024-04-01T00:14:44.568712+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568712+0000","last_undegraded":"2024-04-01T00:14:44.568712+0000","last_fullsized":"2024-04-01T00:14:44.568712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563295+0000","last_change":"2024-04-01T00:14:44.563295+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563295+0000","last_undegraded":"2024-04-01T00:14:44.563295+0000","last_fullsized":"2024-04-01T00:14:44.563295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564385+0000","last_change":"2024-04-01T00:14:44.564385+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564385+0000","last_undegraded":"2024-04-01T00:14:44.564385+0000","last_fullsized":"2024-04-01T00:14:44.564385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566470+0000","last_change":"2024-04-01T00:14:44.566470+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566470+0000","last_undegraded":"2024-04-01T00:14:44.566470+0000","last_fullsized":"2024-04-01T00:14:44.566470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568447+0000","last_change":"2024-04-01T00:14:44.568447+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568447+0000","last_undegraded":"2024-04-01T00:14:44.568447+0000","last_fullsized":"2024-04-01T00:14:44.568447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571511+0000","last_change":"2024-04-01T00:14:44.571511+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571511+0000","last_undegraded":"2024-04-01T00:14:44.571511+0000","last_fullsized":"2024-04-01T00:14:44.571511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563982+0000","last_change":"2024-04-01T00:14:44.563982+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563982+0000","last_undegraded":"2024-04-01T00:14:44.563982+0000","last_fullsized":"2024-04-01T00:14:44.563982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570322+0000","last_change":"2024-04-01T00:14:44.570322+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570322+0000","last_undegraded":"2024-04-01T00:14:44.570322+0000","last_fullsized":"2024-04-01T00:14:44.570322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563281+0000","last_change":"2024-04-01T00:14:44.563281+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563281+0000","last_undegraded":"2024-04-01T00:14:44.563281+0000","last_fullsized":"2024-04-01T00:14:44.563281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565987+0000","last_change":"2024-04-01T00:14:44.565987+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565987+0000","last_undegraded":"2024-04-01T00:14:44.565987+0000","last_fullsized":"2024-04-01T00:14:44.565987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565613+0000","last_change":"2024-04-01T00:14:44.565613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565613+0000","last_undegraded":"2024-04-01T00:14:44.565613+0000","last_fullsized":"2024-04-01T00:14:44.565613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562449+0000","last_change":"2024-04-01T00:14:44.562449+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562449+0000","last_undegraded":"2024-04-01T00:14:44.562449+0000","last_fullsized":"2024-04-01T00:14:44.562449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568007+0000","last_change":"2024-04-01T00:14:44.568007+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568007+0000","last_undegraded":"2024-04-01T00:14:44.568007+0000","last_fullsized":"2024-04-01T00:14:44.568007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568168+0000","last_change":"2024-04-01T00:14:44.568168+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568168+0000","last_undegraded":"2024-04-01T00:14:44.568168+0000","last_fullsized":"2024-04-01T00:14:44.568168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567485+0000","last_change":"2024-04-01T00:14:44.567485+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567485+0000","last_undegraded":"2024-04-01T00:14:44.567485+0000","last_fullsized":"2024-04-01T00:14:44.567485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569340+0000","last_change":"2024-04-01T00:14:44.569340+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569340+0000","last_undegraded":"2024-04-01T00:14:44.569340+0000","last_fullsized":"2024-04-01T00:14:44.569340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565896+0000","last_change":"2024-04-01T00:14:44.565896+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565896+0000","last_undegraded":"2024-04-01T00:14:44.565896+0000","last_fullsized":"2024-04-01T00:14:44.565896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569250+0000","last_change":"2024-04-01T00:14:44.569250+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569250+0000","last_undegraded":"2024-04-01T00:14:44.569250+0000","last_fullsized":"2024-04-01T00:14:44.569250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567609+0000","last_change":"2024-04-01T00:14:44.567609+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567609+0000","last_undegraded":"2024-04-01T00:14:44.567609+0000","last_fullsized":"2024-04-01T00:14:44.567609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567222+0000","last_change":"2024-04-01T00:14:44.567222+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567222+0000","last_undegraded":"2024-04-01T00:14:44.567222+0000","last_fullsized":"2024-04-01T00:14:44.567222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562360+0000","last_change":"2024-04-01T00:14:44.562360+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562360+0000","last_undegraded":"2024-04-01T00:14:44.562360+0000","last_fullsized":"2024-04-01T00:14:44.562360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563629+0000","last_change":"2024-04-01T00:14:44.563629+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563629+0000","last_undegraded":"2024-04-01T00:14:44.563629+0000","last_fullsized":"2024-04-01T00:14:44.563629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566989+0000","last_change":"2024-04-01T00:14:44.566989+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566989+0000","last_undegraded":"2024-04-01T00:14:44.566989+0000","last_fullsized":"2024-04-01T00:14:44.566989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564557+0000","last_change":"2024-04-01T00:14:44.564557+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564557+0000","last_undegraded":"2024-04-01T00:14:44.564557+0000","last_fullsized":"2024-04-01T00:14:44.564557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562736+0000","last_change":"2024-04-01T00:14:44.562736+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562736+0000","last_undegraded":"2024-04-01T00:14:44.562736+0000","last_fullsized":"2024-04-01T00:14:44.562736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568778+0000","last_change":"2024-04-01T00:14:44.568778+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568778+0000","last_undegraded":"2024-04-01T00:14:44.568778+0000","last_fullsized":"2024-04-01T00:14:44.568778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569763+0000","last_change":"2024-04-01T00:14:44.569763+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569763+0000","last_undegraded":"2024-04-01T00:14:44.569763+0000","last_fullsized":"2024-04-01T00:14:44.569763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567935+0000","last_change":"2024-04-01T00:14:44.567935+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567935+0000","last_undegraded":"2024-04-01T00:14:44.567935+0000","last_fullsized":"2024-04-01T00:14:44.567935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571138+0000","last_change":"2024-04-01T00:14:44.571138+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571138+0000","last_undegraded":"2024-04-01T00:14:44.571138+0000","last_fullsized":"2024-04-01T00:14:44.571138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568613+0000","last_change":"2024-04-01T00:14:44.568613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568613+0000","last_undegraded":"2024-04-01T00:14:44.568613+0000","last_fullsized":"2024-04-01T00:14:44.568613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585249+0000","last_change":"2024-04-01T00:14:44.579132+0000","last_active":"2024-04-01T00:14:44.585249+0000","last_peered":"2024-04-01T00:14:44.585249+0000","last_clean":"2024-04-01T00:14:44.585249+0000","last_became_active":"2024-04-01T00:14:44.578796+0000","last_became_peered":"2024-04-01T00:14:44.578796+0000","last_unstale":"2024-04-01T00:14:44.585249+0000","last_undegraded":"2024-04-01T00:14:44.585249+0000","last_fullsized":"2024-04-01T00:14:44.585249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567107+0000","last_change":"2024-04-01T00:14:44.567107+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567107+0000","last_undegraded":"2024-04-01T00:14:44.567107+0000","last_fullsized":"2024-04-01T00:14:44.567107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586107+0000","last_change":"2024-04-01T00:14:44.577986+0000","last_active":"2024-04-01T00:14:44.586107+0000","last_peered":"2024-04-01T00:14:44.586107+0000","last_clean":"2024-04-01T00:14:44.586107+0000","last_became_active":"2024-04-01T00:14:44.577654+0000","last_became_peered":"2024-04-01T00:14:44.577654+0000","last_unstale":"2024-04-01T00:14:44.586107+0000","last_undegraded":"2024-04-01T00:14:44.586107+0000","last_fullsized":"2024-04-01T00:14:44.586107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:27.174967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565065+0000","last_change":"2024-04-01T00:14:44.565065+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565065+0000","last_undegraded":"2024-04-01T00:14:44.565065+0000","last_fullsized":"2024-04-01T00:14:44.565065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583764+0000","last_change":"2024-04-01T00:14:44.579540+0000","last_active":"2024-04-01T00:14:44.583764+0000","last_peered":"2024-04-01T00:14:44.583764+0000","last_clean":"2024-04-01T00:14:44.583764+0000","last_became_active":"2024-04-01T00:14:44.579244+0000","last_became_peered":"2024-04-01T00:14:44.579244+0000","last_unstale":"2024-04-01T00:14:44.583764+0000","last_undegraded":"2024-04-01T00:14:44.583764+0000","last_fullsized":"2024-04-01T00:14:44.583764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563627+0000","last_change":"2024-04-01T00:14:44.563627+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563627+0000","last_undegraded":"2024-04-01T00:14:44.563627+0000","last_fullsized":"2024-04-01T00:14:44.563627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.575359+0000","last_change":"2024-04-01T00:14:44.571485+0000","last_active":"2024-04-01T00:14:44.575359+0000","last_peered":"2024-04-01T00:14:44.575359+0000","last_clean":"2024-04-01T00:14:44.575359+0000","last_became_active":"2024-04-01T00:14:44.571204+0000","last_became_peered":"2024-04-01T00:14:44.571204+0000","last_unstale":"2024-04-01T00:14:44.575359+0000","last_undegraded":"2024-04-01T00:14:44.575359+0000","last_fullsized":"2024-04-01T00:14:44.575359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585527+0000","last_change":"2024-04-01T00:14:44.581975+0000","last_active":"2024-04-01T00:14:44.585527+0000","last_peered":"2024-04-01T00:14:44.585527+0000","last_clean":"2024-04-01T00:14:44.585527+0000","last_became_active":"2024-04-01T00:14:44.581662+0000","last_became_peered":"2024-04-01T00:14:44.581662+0000","last_unstale":"2024-04-01T00:14:44.585527+0000","last_undegraded":"2024-04-01T00:14:44.585527+0000","last_fullsized":"2024-04-01T00:14:44.585527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:13:03.945693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562887+0000","last_change":"2024-04-01T00:14:44.562887+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562887+0000","last_undegraded":"2024-04-01T00:14:44.562887+0000","last_fullsized":"2024-04-01T00:14:44.562887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581059+0000","last_change":"2024-04-01T00:14:44.574617+0000","last_active":"2024-04-01T00:14:44.581059+0000","last_peered":"2024-04-01T00:14:44.581059+0000","last_clean":"2024-04-01T00:14:44.581059+0000","last_became_active":"2024-04-01T00:14:44.574266+0000","last_became_peered":"2024-04-01T00:14:44.574266+0000","last_unstale":"2024-04-01T00:14:44.581059+0000","last_undegraded":"2024-04-01T00:14:44.581059+0000","last_fullsized":"2024-04-01T00:14:44.581059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567715+0000","last_change":"2024-04-01T00:14:44.567715+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567715+0000","last_undegraded":"2024-04-01T00:14:44.567715+0000","last_fullsized":"2024-04-01T00:14:44.567715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583636+0000","last_change":"2024-04-01T00:14:44.577049+0000","last_active":"2024-04-01T00:14:44.583636+0000","last_peered":"2024-04-01T00:14:44.583636+0000","last_clean":"2024-04-01T00:14:44.583636+0000","last_became_active":"2024-04-01T00:14:44.576762+0000","last_became_peered":"2024-04-01T00:14:44.576762+0000","last_unstale":"2024-04-01T00:14:44.583636+0000","last_undegraded":"2024-04-01T00:14:44.583636+0000","last_fullsized":"2024-04-01T00:14:44.583636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563749+0000","last_change":"2024-04-01T00:14:44.563749+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563749+0000","last_undegraded":"2024-04-01T00:14:44.563749+0000","last_fullsized":"2024-04-01T00:14:44.563749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582683+0000","last_change":"2024-04-01T00:14:44.578322+0000","last_active":"2024-04-01T00:14:44.582683+0000","last_peered":"2024-04-01T00:14:44.582683+0000","last_clean":"2024-04-01T00:14:44.582683+0000","last_became_active":"2024-04-01T00:14:44.578036+0000","last_became_peered":"2024-04-01T00:14:44.578036+0000","last_unstale":"2024-04-01T00:14:44.582683+0000","last_undegraded":"2024-04-01T00:14:44.582683+0000","last_fullsized":"2024-04-01T00:14:44.582683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:10:39.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562810+0000","last_change":"2024-04-01T00:14:44.562810+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562810+0000","last_undegraded":"2024-04-01T00:14:44.562810+0000","last_fullsized":"2024-04-01T00:14:44.562810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586393+0000","last_change":"2024-04-01T00:14:44.581137+0000","last_active":"2024-04-01T00:14:44.586393+0000","last_peered":"2024-04-01T00:14:44.586393+0000","last_clean":"2024-04-01T00:14:44.586393+0000","last_became_active":"2024-04-01T00:14:44.580866+0000","last_became_peered":"2024-04-01T00:14:44.580866+0000","last_unstale":"2024-04-01T00:14:44.586393+0000","last_undegraded":"2024-04-01T00:14:44.586393+0000","last_fullsized":"2024-04-01T00:14:44.586393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:03.887331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568759+0000","last_change":"2024-04-01T00:14:44.568759+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568759+0000","last_undegraded":"2024-04-01T00:14:44.568759+0000","last_fullsized":"2024-04-01T00:14:44.568759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586470+0000","last_change":"2024-04-01T00:14:44.581854+0000","last_active":"2024-04-01T00:14:44.586470+0000","last_peered":"2024-04-01T00:14:44.586470+0000","last_clean":"2024-04-01T00:14:44.586470+0000","last_became_active":"2024-04-01T00:14:44.581536+0000","last_became_peered":"2024-04-01T00:14:44.581536+0000","last_unstale":"2024-04-01T00:14:44.586470+0000","last_undegraded":"2024-04-01T00:14:44.586470+0000","last_fullsized":"2024-04-01T00:14:44.586470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566765+0000","last_change":"2024-04-01T00:14:44.566765+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566765+0000","last_undegraded":"2024-04-01T00:14:44.566765+0000","last_fullsized":"2024-04-01T00:14:44.566765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582536+0000","last_change":"2024-04-01T00:14:44.578193+0000","last_active":"2024-04-01T00:14:44.582536+0000","last_peered":"2024-04-01T00:14:44.582536+0000","last_clean":"2024-04-01T00:14:44.582536+0000","last_became_active":"2024-04-01T00:14:44.577675+0000","last_became_peered":"2024-04-01T00:14:44.577675+0000","last_unstale":"2024-04-01T00:14:44.582536+0000","last_undegraded":"2024-04-01T00:14:44.582536+0000","last_fullsized":"2024-04-01T00:14:44.582536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564556+0000","last_change":"2024-04-01T00:14:44.564556+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564556+0000","last_undegraded":"2024-04-01T00:14:44.564556+0000","last_fullsized":"2024-04-01T00:14:44.564556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583762+0000","last_change":"2024-04-01T00:14:44.579376+0000","last_active":"2024-04-01T00:14:44.583762+0000","last_peered":"2024-04-01T00:14:44.583762+0000","last_clean":"2024-04-01T00:14:44.583762+0000","last_became_active":"2024-04-01T00:14:44.579048+0000","last_became_peered":"2024-04-01T00:14:44.579048+0000","last_unstale":"2024-04-01T00:14:44.583762+0000","last_undegraded":"2024-04-01T00:14:44.583762+0000","last_fullsized":"2024-04-01T00:14:44.583762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:29:42.707898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563647+0000","last_change":"2024-04-01T00:14:44.563647+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563647+0000","last_undegraded":"2024-04-01T00:14:44.563647+0000","last_fullsized":"2024-04-01T00:14:44.563647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.580968+0000","last_change":"2024-04-01T00:14:44.573445+0000","last_active":"2024-04-01T00:14:44.580968+0000","last_peered":"2024-04-01T00:14:44.580968+0000","last_clean":"2024-04-01T00:14:44.580968+0000","last_became_active":"2024-04-01T00:14:44.573175+0000","last_became_peered":"2024-04-01T00:14:44.573175+0000","last_unstale":"2024-04-01T00:14:44.580968+0000","last_undegraded":"2024-04-01T00:14:44.580968+0000","last_fullsized":"2024-04-01T00:14:44.580968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:50:51.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567155+0000","last_change":"2024-04-01T00:14:44.567155+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567155+0000","last_undegraded":"2024-04-01T00:14:44.567155+0000","last_fullsized":"2024-04-01T00:14:44.567155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578500+0000","last_change":"2024-04-01T00:14:44.573020+0000","last_active":"2024-04-01T00:14:44.578500+0000","last_peered":"2024-04-01T00:14:44.578500+0000","last_clean":"2024-04-01T00:14:44.578500+0000","last_became_active":"2024-04-01T00:14:44.572756+0000","last_became_peered":"2024-04-01T00:14:44.572756+0000","last_unstale":"2024-04-01T00:14:44.578500+0000","last_undegraded":"2024-04-01T00:14:44.578500+0000","last_fullsized":"2024-04-01T00:14:44.578500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570706+0000","last_change":"2024-04-01T00:14:44.570706+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570706+0000","last_undegraded":"2024-04-01T00:14:44.570706+0000","last_fullsized":"2024-04-01T00:14:44.570706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583807+0000","last_change":"2024-04-01T00:14:44.580598+0000","last_active":"2024-04-01T00:14:44.583807+0000","last_peered":"2024-04-01T00:14:44.583807+0000","last_clean":"2024-04-01T00:14:44.583807+0000","last_became_active":"2024-04-01T00:14:44.580245+0000","last_became_peered":"2024-04-01T00:14:44.580245+0000","last_unstale":"2024-04-01T00:14:44.583807+0000","last_undegraded":"2024-04-01T00:14:44.583807+0000","last_fullsized":"2024-04-01T00:14:44.583807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:31.290107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566150+0000","last_change":"2024-04-01T00:14:44.566150+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566150+0000","last_undegraded":"2024-04-01T00:14:44.566150+0000","last_fullsized":"2024-04-01T00:14:44.566150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583792+0000","last_change":"2024-04-01T00:14:44.580512+0000","last_active":"2024-04-01T00:14:44.583792+0000","last_peered":"2024-04-01T00:14:44.583792+0000","last_clean":"2024-04-01T00:14:44.583792+0000","last_became_active":"2024-04-01T00:14:44.580132+0000","last_became_peered":"2024-04-01T00:14:44.580132+0000","last_unstale":"2024-04-01T00:14:44.583792+0000","last_undegraded":"2024-04-01T00:14:44.583792+0000","last_fullsized":"2024-04-01T00:14:44.583792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569699+0000","last_change":"2024-04-01T00:14:44.569699+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569699+0000","last_undegraded":"2024-04-01T00:14:44.569699+0000","last_fullsized":"2024-04-01T00:14:44.569699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.579022+0000","last_change":"2024-04-01T00:14:44.576879+0000","last_active":"2024-04-01T00:14:44.579022+0000","last_peered":"2024-04-01T00:14:44.579022+0000","last_clean":"2024-04-01T00:14:44.579022+0000","last_became_active":"2024-04-01T00:14:44.576567+0000","last_became_peered":"2024-04-01T00:14:44.576567+0000","last_unstale":"2024-04-01T00:14:44.579022+0000","last_undegraded":"2024-04-01T00:14:44.579022+0000","last_fullsized":"2024-04-01T00:14:44.579022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564049+0000","last_change":"2024-04-01T00:14:44.564049+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564049+0000","last_undegraded":"2024-04-01T00:14:44.564049+0000","last_fullsized":"2024-04-01T00:14:44.564049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581075+0000","last_change":"2024-04-01T00:14:44.574281+0000","last_active":"2024-04-01T00:14:44.581075+0000","last_peered":"2024-04-01T00:14:44.581075+0000","last_clean":"2024-04-01T00:14:44.581075+0000","last_became_active":"2024-04-01T00:14:44.573981+0000","last_became_peered":"2024-04-01T00:14:44.573981+0000","last_unstale":"2024-04-01T00:14:44.581075+0000","last_undegraded":"2024-04-01T00:14:44.581075+0000","last_fullsized":"2024-04-01T00:14:44.581075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:23:57.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563883+0000","last_change":"2024-04-01T00:14:44.563883+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563883+0000","last_undegraded":"2024-04-01T00:14:44.563883+0000","last_fullsized":"2024-04-01T00:14:44.563883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581078+0000","last_change":"2024-04-01T00:14:44.574997+0000","last_active":"2024-04-01T00:14:44.581078+0000","last_peered":"2024-04-01T00:14:44.581078+0000","last_clean":"2024-04-01T00:14:44.581078+0000","last_became_active":"2024-04-01T00:14:44.574734+0000","last_became_peered":"2024-04-01T00:14:44.574734+0000","last_unstale":"2024-04-01T00:14:44.581078+0000","last_undegraded":"2024-04-01T00:14:44.581078+0000","last_fullsized":"2024-04-01T00:14:44.581078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:54:03.085286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565020+0000","last_change":"2024-04-01T00:14:44.565020+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565020+0000","last_undegraded":"2024-04-01T00:14:44.565020+0000","last_fullsized":"2024-04-01T00:14:44.565020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581089+0000","last_change":"2024-04-01T00:14:44.576591+0000","last_active":"2024-04-01T00:14:44.581089+0000","last_peered":"2024-04-01T00:14:44.581089+0000","last_clean":"2024-04-01T00:14:44.581089+0000","last_became_active":"2024-04-01T00:14:44.576336+0000","last_became_peered":"2024-04-01T00:14:44.576336+0000","last_unstale":"2024-04-01T00:14:44.581089+0000","last_undegraded":"2024-04-01T00:14:44.581089+0000","last_fullsized":"2024-04-01T00:14:44.581089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:25:30.722477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586233+0000","last_change":"2024-04-01T00:14:44.579815+0000","last_active":"2024-04-01T00:14:44.586233+0000","last_peered":"2024-04-01T00:14:44.586233+0000","last_clean":"2024-04-01T00:14:44.586233+0000","last_became_active":"2024-04-01T00:14:44.579502+0000","last_became_peered":"2024-04-01T00:14:44.579502+0000","last_unstale":"2024-04-01T00:14:44.586233+0000","last_undegraded":"2024-04-01T00:14:44.586233+0000","last_fullsized":"2024-04-01T00:14:44.586233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565505+0000","last_change":"2024-04-01T00:14:44.565505+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565505+0000","last_undegraded":"2024-04-01T00:14:44.565505+0000","last_fullsized":"2024-04-01T00:14:44.565505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585273+0000","last_change":"2024-04-01T00:14:44.579144+0000","last_active":"2024-04-01T00:14:44.585273+0000","last_peered":"2024-04-01T00:14:44.585273+0000","last_clean":"2024-04-01T00:14:44.585273+0000","last_became_active":"2024-04-01T00:14:44.578861+0000","last_became_peered":"2024-04-01T00:14:44.578861+0000","last_unstale":"2024-04-01T00:14:44.585273+0000","last_undegraded":"2024-04-01T00:14:44.585273+0000","last_fullsized":"2024-04-01T00:14:44.585273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564801+0000","last_change":"2024-04-01T00:14:44.564801+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564801+0000","last_undegraded":"2024-04-01T00:14:44.564801+0000","last_fullsized":"2024-04-01T00:14:44.564801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582727+0000","last_change":"2024-04-01T00:14:44.579438+0000","last_active":"2024-04-01T00:14:44.582727+0000","last_peered":"2024-04-01T00:14:44.582727+0000","last_clean":"2024-04-01T00:14:44.582727+0000","last_became_active":"2024-04-01T00:14:44.579181+0000","last_became_peered":"2024-04-01T00:14:44.579181+0000","last_unstale":"2024-04-01T00:14:44.582727+0000","last_undegraded":"2024-04-01T00:14:44.582727+0000","last_fullsized":"2024-04-01T00:14:44.582727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:51:52.461134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569733+0000","last_change":"2024-04-01T00:14:44.569733+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569733+0000","last_undegraded":"2024-04-01T00:14:44.569733+0000","last_fullsized":"2024-04-01T00:14:44.569733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582543+0000","last_change":"2024-04-01T00:14:44.577757+0000","last_active":"2024-04-01T00:14:44.582543+0000","last_peered":"2024-04-01T00:14:44.582543+0000","last_clean":"2024-04-01T00:14:44.582543+0000","last_became_active":"2024-04-01T00:14:44.577431+0000","last_became_peered":"2024-04-01T00:14:44.577431+0000","last_unstale":"2024-04-01T00:14:44.582543+0000","last_undegraded":"2024-04-01T00:14:44.582543+0000","last_fullsized":"2024-04-01T00:14:44.582543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564992+0000","last_change":"2024-04-01T00:14:44.564992+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564992+0000","last_undegraded":"2024-04-01T00:14:44.564992+0000","last_fullsized":"2024-04-01T00:14:44.564992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.568695+0000","last_change":"2024-04-01T00:14:44.567605+0000","last_active":"2024-04-01T00:14:44.568695+0000","last_peered":"2024-04-01T00:14:44.568695+0000","last_clean":"2024-04-01T00:14:44.568695+0000","last_became_active":"2024-04-01T00:14:44.567238+0000","last_became_peered":"2024-04-01T00:14:44.567238+0000","last_unstale":"2024-04-01T00:14:44.568695+0000","last_undegraded":"2024-04-01T00:14:44.568695+0000","last_fullsized":"2024-04-01T00:14:44.568695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563568+0000","last_change":"2024-04-01T00:14:44.563568+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563568+0000","last_undegraded":"2024-04-01T00:14:44.563568+0000","last_fullsized":"2024-04-01T00:14:44.563568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586191+0000","last_change":"2024-04-01T00:14:44.579442+0000","last_active":"2024-04-01T00:14:44.586191+0000","last_peered":"2024-04-01T00:14:44.586191+0000","last_clean":"2024-04-01T00:14:44.586191+0000","last_became_active":"2024-04-01T00:14:44.579151+0000","last_became_peered":"2024-04-01T00:14:44.579151+0000","last_unstale":"2024-04-01T00:14:44.586191+0000","last_undegraded":"2024-04-01T00:14:44.586191+0000","last_fullsized":"2024-04-01T00:14:44.586191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566707+0000","last_change":"2024-04-01T00:14:44.566707+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566707+0000","last_undegraded":"2024-04-01T00:14:44.566707+0000","last_fullsized":"2024-04-01T00:14:44.566707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.576234+0000","last_change":"2024-04-01T00:14:44.570761+0000","last_active":"2024-04-01T00:14:44.576234+0000","last_peered":"2024-04-01T00:14:44.576234+0000","last_clean":"2024-04-01T00:14:44.576234+0000","last_became_active":"2024-04-01T00:14:44.569379+0000","last_became_peered":"2024-04-01T00:14:44.569379+0000","last_unstale":"2024-04-01T00:14:44.576234+0000","last_undegraded":"2024-04-01T00:14:44.576234+0000","last_fullsized":"2024-04-01T00:14:44.576234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:07.835799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566681+0000","last_change":"2024-04-01T00:14:44.566681+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566681+0000","last_undegraded":"2024-04-01T00:14:44.566681+0000","last_fullsized":"2024-04-01T00:14:44.566681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586132+0000","last_change":"2024-04-01T00:14:44.579166+0000","last_active":"2024-04-01T00:14:44.586132+0000","last_peered":"2024-04-01T00:14:44.586132+0000","last_clean":"2024-04-01T00:14:44.586132+0000","last_became_active":"2024-04-01T00:14:44.578892+0000","last_became_peered":"2024-04-01T00:14:44.578892+0000","last_unstale":"2024-04-01T00:14:44.586132+0000","last_undegraded":"2024-04-01T00:14:44.586132+0000","last_fullsized":"2024-04-01T00:14:44.586132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569677+0000","last_change":"2024-04-01T00:14:44.569677+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569677+0000","last_undegraded":"2024-04-01T00:14:44.569677+0000","last_fullsized":"2024-04-01T00:14:44.569677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.573624+0000","last_change":"2024-04-01T00:14:44.568541+0000","last_active":"2024-04-01T00:14:44.573624+0000","last_peered":"2024-04-01T00:14:44.573624+0000","last_clean":"2024-04-01T00:14:44.573624+0000","last_became_active":"2024-04-01T00:14:44.568266+0000","last_became_peered":"2024-04-01T00:14:44.568266+0000","last_unstale":"2024-04-01T00:14:44.573624+0000","last_undegraded":"2024-04-01T00:14:44.573624+0000","last_fullsized":"2024-04-01T00:14:44.573624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567011+0000","last_change":"2024-04-01T00:14:44.567011+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567011+0000","last_undegraded":"2024-04-01T00:14:44.567011+0000","last_fullsized":"2024-04-01T00:14:44.567011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578489+0000","last_change":"2024-04-01T00:14:44.571915+0000","last_active":"2024-04-01T00:14:44.578489+0000","last_peered":"2024-04-01T00:14:44.578489+0000","last_clean":"2024-04-01T00:14:44.578489+0000","last_became_active":"2024-04-01T00:14:44.571316+0000","last_became_peered":"2024-04-01T00:14:44.571316+0000","last_unstale":"2024-04-01T00:14:44.578489+0000","last_undegraded":"2024-04-01T00:14:44.578489+0000","last_fullsized":"2024-04-01T00:14:44.578489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:40:18.733806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565158+0000","last_change":"2024-04-01T00:14:44.565158+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565158+0000","last_undegraded":"2024-04-01T00:14:44.565158+0000","last_fullsized":"2024-04-01T00:14:44.565158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583719+0000","last_change":"2024-04-01T00:14:44.578180+0000","last_active":"2024-04-01T00:14:44.583719+0000","last_peered":"2024-04-01T00:14:44.583719+0000","last_clean":"2024-04-01T00:14:44.583719+0000","last_became_active":"2024-04-01T00:14:44.577563+0000","last_became_peered":"2024-04-01T00:14:44.577563+0000","last_unstale":"2024-04-01T00:14:44.583719+0000","last_undegraded":"2024-04-01T00:14:44.583719+0000","last_fullsized":"2024-04-01T00:14:44.583719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568248+0000","last_change":"2024-04-01T00:14:44.568248+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568248+0000","last_undegraded":"2024-04-01T00:14:44.568248+0000","last_fullsized":"2024-04-01T00:14:44.568248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.587833+0000","last_change":"2024-04-01T00:14:44.581624+0000","last_active":"2024-04-01T00:14:44.587833+0000","last_peered":"2024-04-01T00:14:44.587833+0000","last_clean":"2024-04-01T00:14:44.587833+0000","last_became_active":"2024-04-01T00:14:44.581339+0000","last_became_peered":"2024-04-01T00:14:44.581339+0000","last_unstale":"2024-04-01T00:14:44.587833+0000","last_undegraded":"2024-04-01T00:14:44.587833+0000","last_fullsized":"2024-04-01T00:14:44.587833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.572619+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:14:44.572619+0000","last_peered":"2024-04-01T00:14:44.572619+0000","last_clean":"2024-04-01T00:14:44.572619+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:14:44.572619+0000","last_undegraded":"2024-04-01T00:14:44.572619+0000","last_fullsized":"2024-04-01T00:14:44.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:14:48.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:14:48.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.716+0000 7f96523d0640 1 Processor -- start 2024-04-01T00:14:48.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.716+0000 7f96523d0640 1 -- start start 2024-04-01T00:14:48.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f96523d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f964c0d08d0 0x7f964c0d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f96523d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f964c0cc870 0x7f964c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f96523d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f964c0d25a0 con 0x7f964c0d1c80 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f96523d0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f964c0b2a40 con 0x7f964c0d08d0 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f96523d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f964c0d0d30 con 0x7f964c0cc870 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f964c0d08d0 0x7f964c0d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f964c0d08d0 0x7f964c0d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59730/0 (socket says 172.21.15.64:59730) 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 -- 172.21.15.64:0/267151465 learned_addr learned my addr 172.21.15.64:0/267151465 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f965096d640 1 --2- 172.21.15.64:0/267151465 >> v2:172.21.15.100:3300/0 conn(0x7f964c0cc870 0x7f964c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 -- 172.21.15.64:0/267151465 >> v2:172.21.15.100:3300/0 conn(0x7f964c0cc870 msgr2=0x7f964c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 --2- 172.21.15.64:0/267151465 >> v2:172.21.15.100:3300/0 conn(0x7f964c0cc870 0x7f964c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 -- 172.21.15.64:0/267151465 >> v1:172.21.15.100:6789/0 conn(0x7f964c0d1c80 legacy=0x7f964c0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.717+0000 7f964b7fe640 1 -- 172.21.15.64:0/267151465 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c0b1e30 con 0x7f964c0d08d0 2024-04-01T00:14:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f964b7fe640 1 --2- 172.21.15.64:0/267151465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d08d0 0x7f964c0d2060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f964c06bb90 tx=0x7f963c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99304c7fff20549 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f964affd640 1 -- 172.21.15.64:0/267151465 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c034070 con 0x7f964c0d08d0 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f964affd640 1 -- 172.21.15.64:0/267151465 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c030860 con 0x7f964c0d08d0 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f964affd640 1 -- 172.21.15.64:0/267151465 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c030c30 con 0x7f964c0d08d0 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f96523d0640 1 -- 172.21.15.64:0/267151465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d08d0 msgr2=0x7f964c0d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.718+0000 7f96523d0640 1 --2- 172.21.15.64:0/267151465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d08d0 0x7f964c0d2060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f964c06bb90 tx=0x7f963c02b9a0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 -- 172.21.15.64:0/267151465 shutdown_connections 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 --2- 172.21.15.64:0/267151465 >> v2:172.21.15.100:3300/0 conn(0x7f964c0cc870 0x7f964c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 --2- 172.21.15.64:0/267151465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d08d0 0x7f964c0d2060 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 -- 172.21.15.64:0/267151465 >> 172.21.15.64:0/267151465 conn(0x7f964c09a980 msgr2=0x7f964c09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 -- 172.21.15.64:0/267151465 shutdown_connections 2024-04-01T00:14:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.719+0000 7f96523d0640 1 -- 172.21.15.64:0/267151465 wait complete. 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.720+0000 7f96523d0640 1 Processor -- start 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.720+0000 7f96523d0640 1 -- start start 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f964c0cc870 0x7f964c15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f964c0d08d0 0x7f964c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0b26d0 con 0x7f964c0d1c80 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0b23f0 con 0x7f964c0cc870 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f964c15c180 con 0x7f964c0d08d0 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59742/0 (socket says 172.21.15.64:59742) 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 -- 172.21.15.64:0/2116886827 learned_addr learned my addr 172.21.15.64:0/2116886827 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f964b7fe640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f964c0d08d0 0x7f964c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f964bfff640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f964c0cc870 0x7f964c15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 -- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f964c0d08d0 msgr2=0x7f964c0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f964c0d08d0 0x7f964c0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 -- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f964c0cc870 msgr2=0x7f964c15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f964c0cc870 0x7f964c15eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 -- 172.21.15.64:0/2116886827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c0c5b20 con 0x7f964c0d1c80 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f965096d640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964400efc0 tx=0x7f964400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644018020 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9644014480 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c0c72b0 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644014870 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.721+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c06c420 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.722+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c0681a0 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.723+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f9644005d10 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.723+0000 7f9648ff9640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 0x7f963806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.723+0000 7f964bfff640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 0x7f963806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.723+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f964408b0a0 con 0x7f964c0d1c80 2024-04-01T00:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.724+0000 7f964bfff640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 0x7f963806de50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9640001780 tx=0x7f9640009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:48.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.728+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f9644022070 con 0x7f964c0d1c80 2024-04-01T00:14:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.906+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f964c0c5670 con 0x7f963806b990 2024-04-01T00:14:48.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.912+0000 7f9648ff9640 1 -- 172.21.15.64:0/2116886827 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f964c0c5670 con 0x7f963806b990 2024-04-01T00:14:48.918 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:48.920 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:14:48.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 msgr2=0x7f963806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 0x7f963806de50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9640001780 tx=0x7f9640009040 comp rx=0 tx=0).stop 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 msgr2=0x7f964c0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964400efc0 tx=0x7f964400c490 comp rx=0 tx=0).stop 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 shutdown_connections 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f963806b990 0x7f963806de50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f964c0d1c80 0x7f964c0c6c80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f964c0d08d0 0x7f964c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 --2- 172.21.15.64:0/2116886827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f964c0cc870 0x7f964c15eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 >> 172.21.15.64:0/2116886827 conn(0x7f964c09a980 msgr2=0x7f964c159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 shutdown_connections 2024-04-01T00:14:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:48.915+0000 7f96523d0640 1 -- 172.21.15.64:0/2116886827 wait complete. 2024-04-01T00:14:48.926 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-01T00:14:48.756037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219968,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754752,"statfs":{"total":773094113280,"available":772868866048,"internally_reserved":0,"allocated":3899392,"data_stored":2211398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200841"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567529+0000","last_change":"2024-04-01T00:14:44.567529+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567529+0000","last_undegraded":"2024-04-01T00:14:44.567529+0000","last_fullsized":"2024-04-01T00:14:44.567529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568712+0000","last_change":"2024-04-01T00:14:44.568712+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568712+0000","last_undegraded":"2024-04-01T00:14:44.568712+0000","last_fullsized":"2024-04-01T00:14:44.568712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563295+0000","last_change":"2024-04-01T00:14:44.563295+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563295+0000","last_undegraded":"2024-04-01T00:14:44.563295+0000","last_fullsized":"2024-04-01T00:14:44.563295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564385+0000","last_change":"2024-04-01T00:14:44.564385+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564385+0000","last_undegraded":"2024-04-01T00:14:44.564385+0000","last_fullsized":"2024-04-01T00:14:44.564385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566470+0000","last_change":"2024-04-01T00:14:44.566470+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566470+0000","last_undegraded":"2024-04-01T00:14:44.566470+0000","last_fullsized":"2024-04-01T00:14:44.566470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568447+0000","last_change":"2024-04-01T00:14:44.568447+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568447+0000","last_undegraded":"2024-04-01T00:14:44.568447+0000","last_fullsized":"2024-04-01T00:14:44.568447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571511+0000","last_change":"2024-04-01T00:14:44.571511+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571511+0000","last_undegraded":"2024-04-01T00:14:44.571511+0000","last_fullsized":"2024-04-01T00:14:44.571511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563982+0000","last_change":"2024-04-01T00:14:44.563982+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563982+0000","last_undegraded":"2024-04-01T00:14:44.563982+0000","last_fullsized":"2024-04-01T00:14:44.563982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570322+0000","last_change":"2024-04-01T00:14:44.570322+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570322+0000","last_undegraded":"2024-04-01T00:14:44.570322+0000","last_fullsized":"2024-04-01T00:14:44.570322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563281+0000","last_change":"2024-04-01T00:14:44.563281+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563281+0000","last_undegraded":"2024-04-01T00:14:44.563281+0000","last_fullsized":"2024-04-01T00:14:44.563281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565987+0000","last_change":"2024-04-01T00:14:44.565987+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565987+0000","last_undegraded":"2024-04-01T00:14:44.565987+0000","last_fullsized":"2024-04-01T00:14:44.565987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565613+0000","last_change":"2024-04-01T00:14:44.565613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565613+0000","last_undegraded":"2024-04-01T00:14:44.565613+0000","last_fullsized":"2024-04-01T00:14:44.565613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562449+0000","last_change":"2024-04-01T00:14:44.562449+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562449+0000","last_undegraded":"2024-04-01T00:14:44.562449+0000","last_fullsized":"2024-04-01T00:14:44.562449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568007+0000","last_change":"2024-04-01T00:14:44.568007+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568007+0000","last_undegraded":"2024-04-01T00:14:44.568007+0000","last_fullsized":"2024-04-01T00:14:44.568007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568168+0000","last_change":"2024-04-01T00:14:44.568168+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568168+0000","last_undegraded":"2024-04-01T00:14:44.568168+0000","last_fullsized":"2024-04-01T00:14:44.568168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567485+0000","last_change":"2024-04-01T00:14:44.567485+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567485+0000","last_undegraded":"2024-04-01T00:14:44.567485+0000","last_fullsized":"2024-04-01T00:14:44.567485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569340+0000","last_change":"2024-04-01T00:14:44.569340+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569340+0000","last_undegraded":"2024-04-01T00:14:44.569340+0000","last_fullsized":"2024-04-01T00:14:44.569340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565896+0000","last_change":"2024-04-01T00:14:44.565896+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565896+0000","last_undegraded":"2024-04-01T00:14:44.565896+0000","last_fullsized":"2024-04-01T00:14:44.565896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569250+0000","last_change":"2024-04-01T00:14:44.569250+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569250+0000","last_undegraded":"2024-04-01T00:14:44.569250+0000","last_fullsized":"2024-04-01T00:14:44.569250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567609+0000","last_change":"2024-04-01T00:14:44.567609+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567609+0000","last_undegraded":"2024-04-01T00:14:44.567609+0000","last_fullsized":"2024-04-01T00:14:44.567609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567222+0000","last_change":"2024-04-01T00:14:44.567222+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567222+0000","last_undegraded":"2024-04-01T00:14:44.567222+0000","last_fullsized":"2024-04-01T00:14:44.567222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562360+0000","last_change":"2024-04-01T00:14:44.562360+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562360+0000","last_undegraded":"2024-04-01T00:14:44.562360+0000","last_fullsized":"2024-04-01T00:14:44.562360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563629+0000","last_change":"2024-04-01T00:14:44.563629+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563629+0000","last_undegraded":"2024-04-01T00:14:44.563629+0000","last_fullsized":"2024-04-01T00:14:44.563629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566989+0000","last_change":"2024-04-01T00:14:44.566989+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566989+0000","last_undegraded":"2024-04-01T00:14:44.566989+0000","last_fullsized":"2024-04-01T00:14:44.566989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564557+0000","last_change":"2024-04-01T00:14:44.564557+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564557+0000","last_undegraded":"2024-04-01T00:14:44.564557+0000","last_fullsized":"2024-04-01T00:14:44.564557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562736+0000","last_change":"2024-04-01T00:14:44.562736+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562736+0000","last_undegraded":"2024-04-01T00:14:44.562736+0000","last_fullsized":"2024-04-01T00:14:44.562736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568778+0000","last_change":"2024-04-01T00:14:44.568778+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568778+0000","last_undegraded":"2024-04-01T00:14:44.568778+0000","last_fullsized":"2024-04-01T00:14:44.568778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569763+0000","last_change":"2024-04-01T00:14:44.569763+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569763+0000","last_undegraded":"2024-04-01T00:14:44.569763+0000","last_fullsized":"2024-04-01T00:14:44.569763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567935+0000","last_change":"2024-04-01T00:14:44.567935+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567935+0000","last_undegraded":"2024-04-01T00:14:44.567935+0000","last_fullsized":"2024-04-01T00:14:44.567935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571138+0000","last_change":"2024-04-01T00:14:44.571138+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571138+0000","last_undegraded":"2024-04-01T00:14:44.571138+0000","last_fullsized":"2024-04-01T00:14:44.571138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568613+0000","last_change":"2024-04-01T00:14:44.568613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568613+0000","last_undegraded":"2024-04-01T00:14:44.568613+0000","last_fullsized":"2024-04-01T00:14:44.568613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585249+0000","last_change":"2024-04-01T00:14:44.579132+0000","last_active":"2024-04-01T00:14:44.585249+0000","last_peered":"2024-04-01T00:14:44.585249+0000","last_clean":"2024-04-01T00:14:44.585249+0000","last_became_active":"2024-04-01T00:14:44.578796+0000","last_became_peered":"2024-04-01T00:14:44.578796+0000","last_unstale":"2024-04-01T00:14:44.585249+0000","last_undegraded":"2024-04-01T00:14:44.585249+0000","last_fullsized":"2024-04-01T00:14:44.585249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567107+0000","last_change":"2024-04-01T00:14:44.567107+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567107+0000","last_undegraded":"2024-04-01T00:14:44.567107+0000","last_fullsized":"2024-04-01T00:14:44.567107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586107+0000","last_change":"2024-04-01T00:14:44.577986+0000","last_active":"2024-04-01T00:14:44.586107+0000","last_peered":"2024-04-01T00:14:44.586107+0000","last_clean":"2024-04-01T00:14:44.586107+0000","last_became_active":"2024-04-01T00:14:44.577654+0000","last_became_peered":"2024-04-01T00:14:44.577654+0000","last_unstale":"2024-04-01T00:14:44.586107+0000","last_undegraded":"2024-04-01T00:14:44.586107+0000","last_fullsized":"2024-04-01T00:14:44.586107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:27.174967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565065+0000","last_change":"2024-04-01T00:14:44.565065+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565065+0000","last_undegraded":"2024-04-01T00:14:44.565065+0000","last_fullsized":"2024-04-01T00:14:44.565065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583764+0000","last_change":"2024-04-01T00:14:44.579540+0000","last_active":"2024-04-01T00:14:44.583764+0000","last_peered":"2024-04-01T00:14:44.583764+0000","last_clean":"2024-04-01T00:14:44.583764+0000","last_became_active":"2024-04-01T00:14:44.579244+0000","last_became_peered":"2024-04-01T00:14:44.579244+0000","last_unstale":"2024-04-01T00:14:44.583764+0000","last_undegraded":"2024-04-01T00:14:44.583764+0000","last_fullsized":"2024-04-01T00:14:44.583764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563627+0000","last_change":"2024-04-01T00:14:44.563627+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563627+0000","last_undegraded":"2024-04-01T00:14:44.563627+0000","last_fullsized":"2024-04-01T00:14:44.563627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.575359+0000","last_change":"2024-04-01T00:14:44.571485+0000","last_active":"2024-04-01T00:14:44.575359+0000","last_peered":"2024-04-01T00:14:44.575359+0000","last_clean":"2024-04-01T00:14:44.575359+0000","last_became_active":"2024-04-01T00:14:44.571204+0000","last_became_peered":"2024-04-01T00:14:44.571204+0000","last_unstale":"2024-04-01T00:14:44.575359+0000","last_undegraded":"2024-04-01T00:14:44.575359+0000","last_fullsized":"2024-04-01T00:14:44.575359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585527+0000","last_change":"2024-04-01T00:14:44.581975+0000","last_active":"2024-04-01T00:14:44.585527+0000","last_peered":"2024-04-01T00:14:44.585527+0000","last_clean":"2024-04-01T00:14:44.585527+0000","last_became_active":"2024-04-01T00:14:44.581662+0000","last_became_peered":"2024-04-01T00:14:44.581662+0000","last_unstale":"2024-04-01T00:14:44.585527+0000","last_undegraded":"2024-04-01T00:14:44.585527+0000","last_fullsized":"2024-04-01T00:14:44.585527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:13:03.945693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562887+0000","last_change":"2024-04-01T00:14:44.562887+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562887+0000","last_undegraded":"2024-04-01T00:14:44.562887+0000","last_fullsized":"2024-04-01T00:14:44.562887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581059+0000","last_change":"2024-04-01T00:14:44.574617+0000","last_active":"2024-04-01T00:14:44.581059+0000","last_peered":"2024-04-01T00:14:44.581059+0000","last_clean":"2024-04-01T00:14:44.581059+0000","last_became_active":"2024-04-01T00:14:44.574266+0000","last_became_peered":"2024-04-01T00:14:44.574266+0000","last_unstale":"2024-04-01T00:14:44.581059+0000","last_undegraded":"2024-04-01T00:14:44.581059+0000","last_fullsized":"2024-04-01T00:14:44.581059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567715+0000","last_change":"2024-04-01T00:14:44.567715+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567715+0000","last_undegraded":"2024-04-01T00:14:44.567715+0000","last_fullsized":"2024-04-01T00:14:44.567715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583636+0000","last_change":"2024-04-01T00:14:44.577049+0000","last_active":"2024-04-01T00:14:44.583636+0000","last_peered":"2024-04-01T00:14:44.583636+0000","last_clean":"2024-04-01T00:14:44.583636+0000","last_became_active":"2024-04-01T00:14:44.576762+0000","last_became_peered":"2024-04-01T00:14:44.576762+0000","last_unstale":"2024-04-01T00:14:44.583636+0000","last_undegraded":"2024-04-01T00:14:44.583636+0000","last_fullsized":"2024-04-01T00:14:44.583636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563749+0000","last_change":"2024-04-01T00:14:44.563749+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563749+0000","last_undegraded":"2024-04-01T00:14:44.563749+0000","last_fullsized":"2024-04-01T00:14:44.563749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582683+0000","last_change":"2024-04-01T00:14:44.578322+0000","last_active":"2024-04-01T00:14:44.582683+0000","last_peered":"2024-04-01T00:14:44.582683+0000","last_clean":"2024-04-01T00:14:44.582683+0000","last_became_active":"2024-04-01T00:14:44.578036+0000","last_became_peered":"2024-04-01T00:14:44.578036+0000","last_unstale":"2024-04-01T00:14:44.582683+0000","last_undegraded":"2024-04-01T00:14:44.582683+0000","last_fullsized":"2024-04-01T00:14:44.582683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:10:39.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562810+0000","last_change":"2024-04-01T00:14:44.562810+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562810+0000","last_undegraded":"2024-04-01T00:14:44.562810+0000","last_fullsized":"2024-04-01T00:14:44.562810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586393+0000","last_change":"2024-04-01T00:14:44.581137+0000","last_active":"2024-04-01T00:14:44.586393+0000","last_peered":"2024-04-01T00:14:44.586393+0000","last_clean":"2024-04-01T00:14:44.586393+0000","last_became_active":"2024-04-01T00:14:44.580866+0000","last_became_peered":"2024-04-01T00:14:44.580866+0000","last_unstale":"2024-04-01T00:14:44.586393+0000","last_undegraded":"2024-04-01T00:14:44.586393+0000","last_fullsized":"2024-04-01T00:14:44.586393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:03.887331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568759+0000","last_change":"2024-04-01T00:14:44.568759+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568759+0000","last_undegraded":"2024-04-01T00:14:44.568759+0000","last_fullsized":"2024-04-01T00:14:44.568759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586470+0000","last_change":"2024-04-01T00:14:44.581854+0000","last_active":"2024-04-01T00:14:44.586470+0000","last_peered":"2024-04-01T00:14:44.586470+0000","last_clean":"2024-04-01T00:14:44.586470+0000","last_became_active":"2024-04-01T00:14:44.581536+0000","last_became_peered":"2024-04-01T00:14:44.581536+0000","last_unstale":"2024-04-01T00:14:44.586470+0000","last_undegraded":"2024-04-01T00:14:44.586470+0000","last_fullsized":"2024-04-01T00:14:44.586470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566765+0000","last_change":"2024-04-01T00:14:44.566765+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566765+0000","last_undegraded":"2024-04-01T00:14:44.566765+0000","last_fullsized":"2024-04-01T00:14:44.566765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582536+0000","last_change":"2024-04-01T00:14:44.578193+0000","last_active":"2024-04-01T00:14:44.582536+0000","last_peered":"2024-04-01T00:14:44.582536+0000","last_clean":"2024-04-01T00:14:44.582536+0000","last_became_active":"2024-04-01T00:14:44.577675+0000","last_became_peered":"2024-04-01T00:14:44.577675+0000","last_unstale":"2024-04-01T00:14:44.582536+0000","last_undegraded":"2024-04-01T00:14:44.582536+0000","last_fullsized":"2024-04-01T00:14:44.582536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564556+0000","last_change":"2024-04-01T00:14:44.564556+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564556+0000","last_undegraded":"2024-04-01T00:14:44.564556+0000","last_fullsized":"2024-04-01T00:14:44.564556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583762+0000","last_change":"2024-04-01T00:14:44.579376+0000","last_active":"2024-04-01T00:14:44.583762+0000","last_peered":"2024-04-01T00:14:44.583762+0000","last_clean":"2024-04-01T00:14:44.583762+0000","last_became_active":"2024-04-01T00:14:44.579048+0000","last_became_peered":"2024-04-01T00:14:44.579048+0000","last_unstale":"2024-04-01T00:14:44.583762+0000","last_undegraded":"2024-04-01T00:14:44.583762+0000","last_fullsized":"2024-04-01T00:14:44.583762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:29:42.707898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563647+0000","last_change":"2024-04-01T00:14:44.563647+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563647+0000","last_undegraded":"2024-04-01T00:14:44.563647+0000","last_fullsized":"2024-04-01T00:14:44.563647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.580968+0000","last_change":"2024-04-01T00:14:44.573445+0000","last_active":"2024-04-01T00:14:44.580968+0000","last_peered":"2024-04-01T00:14:44.580968+0000","last_clean":"2024-04-01T00:14:44.580968+0000","last_became_active":"2024-04-01T00:14:44.573175+0000","last_became_peered":"2024-04-01T00:14:44.573175+0000","last_unstale":"2024-04-01T00:14:44.580968+0000","last_undegraded":"2024-04-01T00:14:44.580968+0000","last_fullsized":"2024-04-01T00:14:44.580968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:50:51.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567155+0000","last_change":"2024-04-01T00:14:44.567155+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567155+0000","last_undegraded":"2024-04-01T00:14:44.567155+0000","last_fullsized":"2024-04-01T00:14:44.567155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578500+0000","last_change":"2024-04-01T00:14:44.573020+0000","last_active":"2024-04-01T00:14:44.578500+0000","last_peered":"2024-04-01T00:14:44.578500+0000","last_clean":"2024-04-01T00:14:44.578500+0000","last_became_active":"2024-04-01T00:14:44.572756+0000","last_became_peered":"2024-04-01T00:14:44.572756+0000","last_unstale":"2024-04-01T00:14:44.578500+0000","last_undegraded":"2024-04-01T00:14:44.578500+0000","last_fullsized":"2024-04-01T00:14:44.578500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570706+0000","last_change":"2024-04-01T00:14:44.570706+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570706+0000","last_undegraded":"2024-04-01T00:14:44.570706+0000","last_fullsized":"2024-04-01T00:14:44.570706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583807+0000","last_change":"2024-04-01T00:14:44.580598+0000","last_active":"2024-04-01T00:14:44.583807+0000","last_peered":"2024-04-01T00:14:44.583807+0000","last_clean":"2024-04-01T00:14:44.583807+0000","last_became_active":"2024-04-01T00:14:44.580245+0000","last_became_peered":"2024-04-01T00:14:44.580245+0000","last_unstale":"2024-04-01T00:14:44.583807+0000","last_undegraded":"2024-04-01T00:14:44.583807+0000","last_fullsized":"2024-04-01T00:14:44.583807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:31.290107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566150+0000","last_change":"2024-04-01T00:14:44.566150+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566150+0000","last_undegraded":"2024-04-01T00:14:44.566150+0000","last_fullsized":"2024-04-01T00:14:44.566150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583792+0000","last_change":"2024-04-01T00:14:44.580512+0000","last_active":"2024-04-01T00:14:44.583792+0000","last_peered":"2024-04-01T00:14:44.583792+0000","last_clean":"2024-04-01T00:14:44.583792+0000","last_became_active":"2024-04-01T00:14:44.580132+0000","last_became_peered":"2024-04-01T00:14:44.580132+0000","last_unstale":"2024-04-01T00:14:44.583792+0000","last_undegraded":"2024-04-01T00:14:44.583792+0000","last_fullsized":"2024-04-01T00:14:44.583792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569699+0000","last_change":"2024-04-01T00:14:44.569699+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569699+0000","last_undegraded":"2024-04-01T00:14:44.569699+0000","last_fullsized":"2024-04-01T00:14:44.569699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.579022+0000","last_change":"2024-04-01T00:14:44.576879+0000","last_active":"2024-04-01T00:14:44.579022+0000","last_peered":"2024-04-01T00:14:44.579022+0000","last_clean":"2024-04-01T00:14:44.579022+0000","last_became_active":"2024-04-01T00:14:44.576567+0000","last_became_peered":"2024-04-01T00:14:44.576567+0000","last_unstale":"2024-04-01T00:14:44.579022+0000","last_undegraded":"2024-04-01T00:14:44.579022+0000","last_fullsized":"2024-04-01T00:14:44.579022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564049+0000","last_change":"2024-04-01T00:14:44.564049+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564049+0000","last_undegraded":"2024-04-01T00:14:44.564049+0000","last_fullsized":"2024-04-01T00:14:44.564049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581075+0000","last_change":"2024-04-01T00:14:44.574281+0000","last_active":"2024-04-01T00:14:44.581075+0000","last_peered":"2024-04-01T00:14:44.581075+0000","last_clean":"2024-04-01T00:14:44.581075+0000","last_became_active":"2024-04-01T00:14:44.573981+0000","last_became_peered":"2024-04-01T00:14:44.573981+0000","last_unstale":"2024-04-01T00:14:44.581075+0000","last_undegraded":"2024-04-01T00:14:44.581075+0000","last_fullsized":"2024-04-01T00:14:44.581075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:23:57.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563883+0000","last_change":"2024-04-01T00:14:44.563883+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563883+0000","last_undegraded":"2024-04-01T00:14:44.563883+0000","last_fullsized":"2024-04-01T00:14:44.563883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581078+0000","last_change":"2024-04-01T00:14:44.574997+0000","last_active":"2024-04-01T00:14:44.581078+0000","last_peered":"2024-04-01T00:14:44.581078+0000","last_clean":"2024-04-01T00:14:44.581078+0000","last_became_active":"2024-04-01T00:14:44.574734+0000","last_became_peered":"2024-04-01T00:14:44.574734+0000","last_unstale":"2024-04-01T00:14:44.581078+0000","last_undegraded":"2024-04-01T00:14:44.581078+0000","last_fullsized":"2024-04-01T00:14:44.581078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:54:03.085286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565020+0000","last_change":"2024-04-01T00:14:44.565020+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565020+0000","last_undegraded":"2024-04-01T00:14:44.565020+0000","last_fullsized":"2024-04-01T00:14:44.565020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581089+0000","last_change":"2024-04-01T00:14:44.576591+0000","last_active":"2024-04-01T00:14:44.581089+0000","last_peered":"2024-04-01T00:14:44.581089+0000","last_clean":"2024-04-01T00:14:44.581089+0000","last_became_active":"2024-04-01T00:14:44.576336+0000","last_became_peered":"2024-04-01T00:14:44.576336+0000","last_unstale":"2024-04-01T00:14:44.581089+0000","last_undegraded":"2024-04-01T00:14:44.581089+0000","last_fullsized":"2024-04-01T00:14:44.581089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:25:30.722477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586233+0000","last_change":"2024-04-01T00:14:44.579815+0000","last_active":"2024-04-01T00:14:44.586233+0000","last_peered":"2024-04-01T00:14:44.586233+0000","last_clean":"2024-04-01T00:14:44.586233+0000","last_became_active":"2024-04-01T00:14:44.579502+0000","last_became_peered":"2024-04-01T00:14:44.579502+0000","last_unstale":"2024-04-01T00:14:44.586233+0000","last_undegraded":"2024-04-01T00:14:44.586233+0000","last_fullsized":"2024-04-01T00:14:44.586233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565505+0000","last_change":"2024-04-01T00:14:44.565505+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565505+0000","last_undegraded":"2024-04-01T00:14:44.565505+0000","last_fullsized":"2024-04-01T00:14:44.565505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585273+0000","last_change":"2024-04-01T00:14:44.579144+0000","last_active":"2024-04-01T00:14:44.585273+0000","last_peered":"2024-04-01T00:14:44.585273+0000","last_clean":"2024-04-01T00:14:44.585273+0000","last_became_active":"2024-04-01T00:14:44.578861+0000","last_became_peered":"2024-04-01T00:14:44.578861+0000","last_unstale":"2024-04-01T00:14:44.585273+0000","last_undegraded":"2024-04-01T00:14:44.585273+0000","last_fullsized":"2024-04-01T00:14:44.585273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564801+0000","last_change":"2024-04-01T00:14:44.564801+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564801+0000","last_undegraded":"2024-04-01T00:14:44.564801+0000","last_fullsized":"2024-04-01T00:14:44.564801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582727+0000","last_change":"2024-04-01T00:14:44.579438+0000","last_active":"2024-04-01T00:14:44.582727+0000","last_peered":"2024-04-01T00:14:44.582727+0000","last_clean":"2024-04-01T00:14:44.582727+0000","last_became_active":"2024-04-01T00:14:44.579181+0000","last_became_peered":"2024-04-01T00:14:44.579181+0000","last_unstale":"2024-04-01T00:14:44.582727+0000","last_undegraded":"2024-04-01T00:14:44.582727+0000","last_fullsized":"2024-04-01T00:14:44.582727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:51:52.461134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569733+0000","last_change":"2024-04-01T00:14:44.569733+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569733+0000","last_undegraded":"2024-04-01T00:14:44.569733+0000","last_fullsized":"2024-04-01T00:14:44.569733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582543+0000","last_change":"2024-04-01T00:14:44.577757+0000","last_active":"2024-04-01T00:14:44.582543+0000","last_peered":"2024-04-01T00:14:44.582543+0000","last_clean":"2024-04-01T00:14:44.582543+0000","last_became_active":"2024-04-01T00:14:44.577431+0000","last_became_peered":"2024-04-01T00:14:44.577431+0000","last_unstale":"2024-04-01T00:14:44.582543+0000","last_undegraded":"2024-04-01T00:14:44.582543+0000","last_fullsized":"2024-04-01T00:14:44.582543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564992+0000","last_change":"2024-04-01T00:14:44.564992+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564992+0000","last_undegraded":"2024-04-01T00:14:44.564992+0000","last_fullsized":"2024-04-01T00:14:44.564992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.568695+0000","last_change":"2024-04-01T00:14:44.567605+0000","last_active":"2024-04-01T00:14:44.568695+0000","last_peered":"2024-04-01T00:14:44.568695+0000","last_clean":"2024-04-01T00:14:44.568695+0000","last_became_active":"2024-04-01T00:14:44.567238+0000","last_became_peered":"2024-04-01T00:14:44.567238+0000","last_unstale":"2024-04-01T00:14:44.568695+0000","last_undegraded":"2024-04-01T00:14:44.568695+0000","last_fullsized":"2024-04-01T00:14:44.568695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563568+0000","last_change":"2024-04-01T00:14:44.563568+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563568+0000","last_undegraded":"2024-04-01T00:14:44.563568+0000","last_fullsized":"2024-04-01T00:14:44.563568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586191+0000","last_change":"2024-04-01T00:14:44.579442+0000","last_active":"2024-04-01T00:14:44.586191+0000","last_peered":"2024-04-01T00:14:44.586191+0000","last_clean":"2024-04-01T00:14:44.586191+0000","last_became_active":"2024-04-01T00:14:44.579151+0000","last_became_peered":"2024-04-01T00:14:44.579151+0000","last_unstale":"2024-04-01T00:14:44.586191+0000","last_undegraded":"2024-04-01T00:14:44.586191+0000","last_fullsized":"2024-04-01T00:14:44.586191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566707+0000","last_change":"2024-04-01T00:14:44.566707+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566707+0000","last_undegraded":"2024-04-01T00:14:44.566707+0000","last_fullsized":"2024-04-01T00:14:44.566707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.576234+0000","last_change":"2024-04-01T00:14:44.570761+0000","last_active":"2024-04-01T00:14:44.576234+0000","last_peered":"2024-04-01T00:14:44.576234+0000","last_clean":"2024-04-01T00:14:44.576234+0000","last_became_active":"2024-04-01T00:14:44.569379+0000","last_became_peered":"2024-04-01T00:14:44.569379+0000","last_unstale":"2024-04-01T00:14:44.576234+0000","last_undegraded":"2024-04-01T00:14:44.576234+0000","last_fullsized":"2024-04-01T00:14:44.576234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:07.835799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566681+0000","last_change":"2024-04-01T00:14:44.566681+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566681+0000","last_undegraded":"2024-04-01T00:14:44.566681+0000","last_fullsized":"2024-04-01T00:14:44.566681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586132+0000","last_change":"2024-04-01T00:14:44.579166+0000","last_active":"2024-04-01T00:14:44.586132+0000","last_peered":"2024-04-01T00:14:44.586132+0000","last_clean":"2024-04-01T00:14:44.586132+0000","last_became_active":"2024-04-01T00:14:44.578892+0000","last_became_peered":"2024-04-01T00:14:44.578892+0000","last_unstale":"2024-04-01T00:14:44.586132+0000","last_undegraded":"2024-04-01T00:14:44.586132+0000","last_fullsized":"2024-04-01T00:14:44.586132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569677+0000","last_change":"2024-04-01T00:14:44.569677+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569677+0000","last_undegraded":"2024-04-01T00:14:44.569677+0000","last_fullsized":"2024-04-01T00:14:44.569677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.573624+0000","last_change":"2024-04-01T00:14:44.568541+0000","last_active":"2024-04-01T00:14:44.573624+0000","last_peered":"2024-04-01T00:14:44.573624+0000","last_clean":"2024-04-01T00:14:44.573624+0000","last_became_active":"2024-04-01T00:14:44.568266+0000","last_became_peered":"2024-04-01T00:14:44.568266+0000","last_unstale":"2024-04-01T00:14:44.573624+0000","last_undegraded":"2024-04-01T00:14:44.573624+0000","last_fullsized":"2024-04-01T00:14:44.573624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567011+0000","last_change":"2024-04-01T00:14:44.567011+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567011+0000","last_undegraded":"2024-04-01T00:14:44.567011+0000","last_fullsized":"2024-04-01T00:14:44.567011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578489+0000","last_change":"2024-04-01T00:14:44.571915+0000","last_active":"2024-04-01T00:14:44.578489+0000","last_peered":"2024-04-01T00:14:44.578489+0000","last_clean":"2024-04-01T00:14:44.578489+0000","last_became_active":"2024-04-01T00:14:44.571316+0000","last_became_peered":"2024-04-01T00:14:44.571316+0000","last_unstale":"2024-04-01T00:14:44.578489+0000","last_undegraded":"2024-04-01T00:14:44.578489+0000","last_fullsized":"2024-04-01T00:14:44.578489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:40:18.733806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565158+0000","last_change":"2024-04-01T00:14:44.565158+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565158+0000","last_undegraded":"2024-04-01T00:14:44.565158+0000","last_fullsized":"2024-04-01T00:14:44.565158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583719+0000","last_change":"2024-04-01T00:14:44.578180+0000","last_active":"2024-04-01T00:14:44.583719+0000","last_peered":"2024-04-01T00:14:44.583719+0000","last_clean":"2024-04-01T00:14:44.583719+0000","last_became_active":"2024-04-01T00:14:44.577563+0000","last_became_peered":"2024-04-01T00:14:44.577563+0000","last_unstale":"2024-04-01T00:14:44.583719+0000","last_undegraded":"2024-04-01T00:14:44.583719+0000","last_fullsized":"2024-04-01T00:14:44.583719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568248+0000","last_change":"2024-04-01T00:14:44.568248+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568248+0000","last_undegraded":"2024-04-01T00:14:44.568248+0000","last_fullsized":"2024-04-01T00:14:44.568248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.587833+0000","last_change":"2024-04-01T00:14:44.581624+0000","last_active":"2024-04-01T00:14:44.587833+0000","last_peered":"2024-04-01T00:14:44.587833+0000","last_clean":"2024-04-01T00:14:44.587833+0000","last_became_active":"2024-04-01T00:14:44.581339+0000","last_became_peered":"2024-04-01T00:14:44.581339+0000","last_unstale":"2024-04-01T00:14:44.587833+0000","last_undegraded":"2024-04-01T00:14:44.587833+0000","last_fullsized":"2024-04-01T00:14:44.587833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.572619+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:14:44.572619+0000","last_peered":"2024-04-01T00:14:44.572619+0000","last_clean":"2024-04-01T00:14:44.572619+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:14:44.572619+0000","last_undegraded":"2024-04-01T00:14:44.572619+0000","last_fullsized":"2024-04-01T00:14:44.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:14:48.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:14:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.068+0000 7f4de56ae640 1 Processor -- start 2024-04-01T00:14:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.068+0000 7f4de56ae640 1 -- start start 2024-04-01T00:14:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4de56ae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 0x7f4de00d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4de56ae640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4de00cedb0 0x7f4de00cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4de56ae640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de015c350 con 0x7f4de00cd3f0 2024-04-01T00:14:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4de56ae640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4de00cf8a0 con 0x7f4de00cedb0 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4de56ae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4de00cfb80 con 0x7f4de015bf70 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4ddf7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4de00cedb0 0x7f4de00cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4dde7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4de00cd3f0 0x7f4de00ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:48758/0 (socket says 172.21.15.64:48758) 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4dde7fc640 1 -- 172.21.15.64:0/2995375128 learned_addr learned my addr 172.21.15.64:0/2995375128 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.069+0000 7f4ddeffd640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 0x7f4de00d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddf7fe640 1 -- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 msgr2=0x7f4de00d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddf7fe640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 0x7f4de00d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddf7fe640 1 -- 172.21.15.64:0/2995375128 >> v1:172.21.15.100:6789/0 conn(0x7f4de00cd3f0 legacy=0x7f4de00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddf7fe640 1 -- 172.21.15.64:0/2995375128 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de00cfe60 con 0x7f4de00cedb0 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4dddffb640 1 -- 172.21.15.64:0/2995375128 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377162484 0 0) 0x7f4de015c350 con 0x7f4de00cd3f0 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddeffd640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 0x7f4de00d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4ddf7fe640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de00cedb0 0x7f4de00cd870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009d90 tx=0x7f4dd002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d7fac133c0e22a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.070+0000 7f4dddffb640 1 -- 172.21.15.64:0/2995375128 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0039070 con 0x7f4de00cedb0 2024-04-01T00:14:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4dddffb640 1 -- 172.21.15.64:0/2995375128 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd002cc50 con 0x7f4de00cedb0 2024-04-01T00:14:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4dddffb640 1 -- 172.21.15.64:0/2995375128 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0034440 con 0x7f4de00cedb0 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 -- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de00cedb0 msgr2=0x7f4de00cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de00cedb0 0x7f4de00cd870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009d90 tx=0x7f4dd002bea0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 -- 172.21.15.64:0/2995375128 shutdown_connections 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de00cedb0 0x7f4de00cd870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 --2- 172.21.15.64:0/2995375128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de015bf70 0x7f4de00d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 -- 172.21.15.64:0/2995375128 >> 172.21.15.64:0/2995375128 conn(0x7f4de00d1cd0 msgr2=0x7f4de00d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 -- 172.21.15.64:0/2995375128 shutdown_connections 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.071+0000 7f4de56ae640 1 -- 172.21.15.64:0/2995375128 wait complete. 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.072+0000 7f4de56ae640 1 Processor -- start 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.072+0000 7f4de56ae640 1 -- start start 2024-04-01T00:14:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de00cedb0 0x7f4de0152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de015bf70 0x7f4de00c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4de00d29a0 con 0x7f4de00cd3f0 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4de00d0420 con 0x7f4de015bf70 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4de56ae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4de00d2b20 con 0x7f4de00cedb0 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4ddeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4ddeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59756/0 (socket says 172.21.15.64:59756) 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4ddeffd640 1 -- 172.21.15.64:0/208828093 learned_addr learned my addr 172.21.15.64:0/208828093 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4ddf7fe640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de015bf70 0x7f4de00c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.073+0000 7f4dde7fc640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de00cedb0 0x7f4de0152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 -- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de00cedb0 msgr2=0x7f4de0152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de00cedb0 0x7f4de0152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 -- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de015bf70 msgr2=0x7f4de00c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de015bf70 0x7f4de00c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 -- 172.21.15.64:0/208828093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de009cd70 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4ddeffd640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ebe0 tx=0x7f4dd800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8017070 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd800cc30 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de01f4f80 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8013470 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.074+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de01f5540 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.075+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de00681a0 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.075+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f4dd801a070 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.076+0000 7f4dc77fe640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 0x7f4dcc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.076+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4dd808a6f0 con 0x7f4de00cd3f0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.076+0000 7f4dde7fc640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 0x7f4dcc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.076+0000 7f4dde7fc640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 0x7f4dcc06de80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4de00b2c40 tx=0x7f4dd4007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.080+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4dd8058cb0 con 0x7f4de00cd3f0 2024-04-01T00:14:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.257+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4de0152690 con 0x7f4dcc06b9c0 2024-04-01T00:14:49.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.264+0000 7f4dc77fe640 1 -- 172.21.15.64:0/208828093 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f4de0152690 con 0x7f4dcc06b9c0 2024-04-01T00:14:49.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:14:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 msgr2=0x7f4dcc06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 0x7f4dcc06de80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4de00b2c40 tx=0x7f4dd4007de0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 msgr2=0x7f4de009c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ebe0 tx=0x7f4dd800c520 comp rx=0 tx=0).stop 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 shutdown_connections 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f4dcc06b9c0 0x7f4dcc06de80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4de015bf70 0x7f4de00c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4de00cedb0 0x7f4de0152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 --2- 172.21.15.64:0/208828093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de00cd3f0 0x7f4de009c830 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 >> 172.21.15.64:0/208828093 conn(0x7f4de00d1cd0 msgr2=0x7f4de00701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 shutdown_connections 2024-04-01T00:14:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.266+0000 7f4de56ae640 1 -- 172.21.15.64:0/208828093 wait complete. 2024-04-01T00:14:49.278 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-01T00:14:48.756037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219968,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754752,"statfs":{"total":773094113280,"available":772868866048,"internally_reserved":0,"allocated":3899392,"data_stored":2211398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200841"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567529+0000","last_change":"2024-04-01T00:14:44.567529+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567529+0000","last_undegraded":"2024-04-01T00:14:44.567529+0000","last_fullsized":"2024-04-01T00:14:44.567529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568712+0000","last_change":"2024-04-01T00:14:44.568712+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568712+0000","last_undegraded":"2024-04-01T00:14:44.568712+0000","last_fullsized":"2024-04-01T00:14:44.568712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563295+0000","last_change":"2024-04-01T00:14:44.563295+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563295+0000","last_undegraded":"2024-04-01T00:14:44.563295+0000","last_fullsized":"2024-04-01T00:14:44.563295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564385+0000","last_change":"2024-04-01T00:14:44.564385+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564385+0000","last_undegraded":"2024-04-01T00:14:44.564385+0000","last_fullsized":"2024-04-01T00:14:44.564385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566470+0000","last_change":"2024-04-01T00:14:44.566470+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566470+0000","last_undegraded":"2024-04-01T00:14:44.566470+0000","last_fullsized":"2024-04-01T00:14:44.566470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568447+0000","last_change":"2024-04-01T00:14:44.568447+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568447+0000","last_undegraded":"2024-04-01T00:14:44.568447+0000","last_fullsized":"2024-04-01T00:14:44.568447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571511+0000","last_change":"2024-04-01T00:14:44.571511+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571511+0000","last_undegraded":"2024-04-01T00:14:44.571511+0000","last_fullsized":"2024-04-01T00:14:44.571511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563982+0000","last_change":"2024-04-01T00:14:44.563982+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563982+0000","last_undegraded":"2024-04-01T00:14:44.563982+0000","last_fullsized":"2024-04-01T00:14:44.563982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570322+0000","last_change":"2024-04-01T00:14:44.570322+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570322+0000","last_undegraded":"2024-04-01T00:14:44.570322+0000","last_fullsized":"2024-04-01T00:14:44.570322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563281+0000","last_change":"2024-04-01T00:14:44.563281+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563281+0000","last_undegraded":"2024-04-01T00:14:44.563281+0000","last_fullsized":"2024-04-01T00:14:44.563281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565987+0000","last_change":"2024-04-01T00:14:44.565987+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565987+0000","last_undegraded":"2024-04-01T00:14:44.565987+0000","last_fullsized":"2024-04-01T00:14:44.565987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565613+0000","last_change":"2024-04-01T00:14:44.565613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565613+0000","last_undegraded":"2024-04-01T00:14:44.565613+0000","last_fullsized":"2024-04-01T00:14:44.565613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562449+0000","last_change":"2024-04-01T00:14:44.562449+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562449+0000","last_undegraded":"2024-04-01T00:14:44.562449+0000","last_fullsized":"2024-04-01T00:14:44.562449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568007+0000","last_change":"2024-04-01T00:14:44.568007+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568007+0000","last_undegraded":"2024-04-01T00:14:44.568007+0000","last_fullsized":"2024-04-01T00:14:44.568007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568168+0000","last_change":"2024-04-01T00:14:44.568168+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568168+0000","last_undegraded":"2024-04-01T00:14:44.568168+0000","last_fullsized":"2024-04-01T00:14:44.568168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567485+0000","last_change":"2024-04-01T00:14:44.567485+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567485+0000","last_undegraded":"2024-04-01T00:14:44.567485+0000","last_fullsized":"2024-04-01T00:14:44.567485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569340+0000","last_change":"2024-04-01T00:14:44.569340+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569340+0000","last_undegraded":"2024-04-01T00:14:44.569340+0000","last_fullsized":"2024-04-01T00:14:44.569340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565896+0000","last_change":"2024-04-01T00:14:44.565896+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565896+0000","last_undegraded":"2024-04-01T00:14:44.565896+0000","last_fullsized":"2024-04-01T00:14:44.565896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569250+0000","last_change":"2024-04-01T00:14:44.569250+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569250+0000","last_undegraded":"2024-04-01T00:14:44.569250+0000","last_fullsized":"2024-04-01T00:14:44.569250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567609+0000","last_change":"2024-04-01T00:14:44.567609+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567609+0000","last_undegraded":"2024-04-01T00:14:44.567609+0000","last_fullsized":"2024-04-01T00:14:44.567609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567222+0000","last_change":"2024-04-01T00:14:44.567222+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567222+0000","last_undegraded":"2024-04-01T00:14:44.567222+0000","last_fullsized":"2024-04-01T00:14:44.567222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562360+0000","last_change":"2024-04-01T00:14:44.562360+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562360+0000","last_undegraded":"2024-04-01T00:14:44.562360+0000","last_fullsized":"2024-04-01T00:14:44.562360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563629+0000","last_change":"2024-04-01T00:14:44.563629+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563629+0000","last_undegraded":"2024-04-01T00:14:44.563629+0000","last_fullsized":"2024-04-01T00:14:44.563629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566989+0000","last_change":"2024-04-01T00:14:44.566989+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566989+0000","last_undegraded":"2024-04-01T00:14:44.566989+0000","last_fullsized":"2024-04-01T00:14:44.566989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564557+0000","last_change":"2024-04-01T00:14:44.564557+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564557+0000","last_undegraded":"2024-04-01T00:14:44.564557+0000","last_fullsized":"2024-04-01T00:14:44.564557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562736+0000","last_change":"2024-04-01T00:14:44.562736+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562736+0000","last_undegraded":"2024-04-01T00:14:44.562736+0000","last_fullsized":"2024-04-01T00:14:44.562736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568778+0000","last_change":"2024-04-01T00:14:44.568778+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568778+0000","last_undegraded":"2024-04-01T00:14:44.568778+0000","last_fullsized":"2024-04-01T00:14:44.568778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569763+0000","last_change":"2024-04-01T00:14:44.569763+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569763+0000","last_undegraded":"2024-04-01T00:14:44.569763+0000","last_fullsized":"2024-04-01T00:14:44.569763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567935+0000","last_change":"2024-04-01T00:14:44.567935+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567935+0000","last_undegraded":"2024-04-01T00:14:44.567935+0000","last_fullsized":"2024-04-01T00:14:44.567935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571138+0000","last_change":"2024-04-01T00:14:44.571138+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571138+0000","last_undegraded":"2024-04-01T00:14:44.571138+0000","last_fullsized":"2024-04-01T00:14:44.571138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568613+0000","last_change":"2024-04-01T00:14:44.568613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568613+0000","last_undegraded":"2024-04-01T00:14:44.568613+0000","last_fullsized":"2024-04-01T00:14:44.568613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585249+0000","last_change":"2024-04-01T00:14:44.579132+0000","last_active":"2024-04-01T00:14:44.585249+0000","last_peered":"2024-04-01T00:14:44.585249+0000","last_clean":"2024-04-01T00:14:44.585249+0000","last_became_active":"2024-04-01T00:14:44.578796+0000","last_became_peered":"2024-04-01T00:14:44.578796+0000","last_unstale":"2024-04-01T00:14:44.585249+0000","last_undegraded":"2024-04-01T00:14:44.585249+0000","last_fullsized":"2024-04-01T00:14:44.585249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567107+0000","last_change":"2024-04-01T00:14:44.567107+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567107+0000","last_undegraded":"2024-04-01T00:14:44.567107+0000","last_fullsized":"2024-04-01T00:14:44.567107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586107+0000","last_change":"2024-04-01T00:14:44.577986+0000","last_active":"2024-04-01T00:14:44.586107+0000","last_peered":"2024-04-01T00:14:44.586107+0000","last_clean":"2024-04-01T00:14:44.586107+0000","last_became_active":"2024-04-01T00:14:44.577654+0000","last_became_peered":"2024-04-01T00:14:44.577654+0000","last_unstale":"2024-04-01T00:14:44.586107+0000","last_undegraded":"2024-04-01T00:14:44.586107+0000","last_fullsized":"2024-04-01T00:14:44.586107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:27.174967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565065+0000","last_change":"2024-04-01T00:14:44.565065+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565065+0000","last_undegraded":"2024-04-01T00:14:44.565065+0000","last_fullsized":"2024-04-01T00:14:44.565065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583764+0000","last_change":"2024-04-01T00:14:44.579540+0000","last_active":"2024-04-01T00:14:44.583764+0000","last_peered":"2024-04-01T00:14:44.583764+0000","last_clean":"2024-04-01T00:14:44.583764+0000","last_became_active":"2024-04-01T00:14:44.579244+0000","last_became_peered":"2024-04-01T00:14:44.579244+0000","last_unstale":"2024-04-01T00:14:44.583764+0000","last_undegraded":"2024-04-01T00:14:44.583764+0000","last_fullsized":"2024-04-01T00:14:44.583764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563627+0000","last_change":"2024-04-01T00:14:44.563627+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563627+0000","last_undegraded":"2024-04-01T00:14:44.563627+0000","last_fullsized":"2024-04-01T00:14:44.563627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.575359+0000","last_change":"2024-04-01T00:14:44.571485+0000","last_active":"2024-04-01T00:14:44.575359+0000","last_peered":"2024-04-01T00:14:44.575359+0000","last_clean":"2024-04-01T00:14:44.575359+0000","last_became_active":"2024-04-01T00:14:44.571204+0000","last_became_peered":"2024-04-01T00:14:44.571204+0000","last_unstale":"2024-04-01T00:14:44.575359+0000","last_undegraded":"2024-04-01T00:14:44.575359+0000","last_fullsized":"2024-04-01T00:14:44.575359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585527+0000","last_change":"2024-04-01T00:14:44.581975+0000","last_active":"2024-04-01T00:14:44.585527+0000","last_peered":"2024-04-01T00:14:44.585527+0000","last_clean":"2024-04-01T00:14:44.585527+0000","last_became_active":"2024-04-01T00:14:44.581662+0000","last_became_peered":"2024-04-01T00:14:44.581662+0000","last_unstale":"2024-04-01T00:14:44.585527+0000","last_undegraded":"2024-04-01T00:14:44.585527+0000","last_fullsized":"2024-04-01T00:14:44.585527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:13:03.945693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562887+0000","last_change":"2024-04-01T00:14:44.562887+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562887+0000","last_undegraded":"2024-04-01T00:14:44.562887+0000","last_fullsized":"2024-04-01T00:14:44.562887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581059+0000","last_change":"2024-04-01T00:14:44.574617+0000","last_active":"2024-04-01T00:14:44.581059+0000","last_peered":"2024-04-01T00:14:44.581059+0000","last_clean":"2024-04-01T00:14:44.581059+0000","last_became_active":"2024-04-01T00:14:44.574266+0000","last_became_peered":"2024-04-01T00:14:44.574266+0000","last_unstale":"2024-04-01T00:14:44.581059+0000","last_undegraded":"2024-04-01T00:14:44.581059+0000","last_fullsized":"2024-04-01T00:14:44.581059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567715+0000","last_change":"2024-04-01T00:14:44.567715+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567715+0000","last_undegraded":"2024-04-01T00:14:44.567715+0000","last_fullsized":"2024-04-01T00:14:44.567715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583636+0000","last_change":"2024-04-01T00:14:44.577049+0000","last_active":"2024-04-01T00:14:44.583636+0000","last_peered":"2024-04-01T00:14:44.583636+0000","last_clean":"2024-04-01T00:14:44.583636+0000","last_became_active":"2024-04-01T00:14:44.576762+0000","last_became_peered":"2024-04-01T00:14:44.576762+0000","last_unstale":"2024-04-01T00:14:44.583636+0000","last_undegraded":"2024-04-01T00:14:44.583636+0000","last_fullsized":"2024-04-01T00:14:44.583636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563749+0000","last_change":"2024-04-01T00:14:44.563749+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563749+0000","last_undegraded":"2024-04-01T00:14:44.563749+0000","last_fullsized":"2024-04-01T00:14:44.563749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582683+0000","last_change":"2024-04-01T00:14:44.578322+0000","last_active":"2024-04-01T00:14:44.582683+0000","last_peered":"2024-04-01T00:14:44.582683+0000","last_clean":"2024-04-01T00:14:44.582683+0000","last_became_active":"2024-04-01T00:14:44.578036+0000","last_became_peered":"2024-04-01T00:14:44.578036+0000","last_unstale":"2024-04-01T00:14:44.582683+0000","last_undegraded":"2024-04-01T00:14:44.582683+0000","last_fullsized":"2024-04-01T00:14:44.582683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:10:39.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562810+0000","last_change":"2024-04-01T00:14:44.562810+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562810+0000","last_undegraded":"2024-04-01T00:14:44.562810+0000","last_fullsized":"2024-04-01T00:14:44.562810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586393+0000","last_change":"2024-04-01T00:14:44.581137+0000","last_active":"2024-04-01T00:14:44.586393+0000","last_peered":"2024-04-01T00:14:44.586393+0000","last_clean":"2024-04-01T00:14:44.586393+0000","last_became_active":"2024-04-01T00:14:44.580866+0000","last_became_peered":"2024-04-01T00:14:44.580866+0000","last_unstale":"2024-04-01T00:14:44.586393+0000","last_undegraded":"2024-04-01T00:14:44.586393+0000","last_fullsized":"2024-04-01T00:14:44.586393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:03.887331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568759+0000","last_change":"2024-04-01T00:14:44.568759+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568759+0000","last_undegraded":"2024-04-01T00:14:44.568759+0000","last_fullsized":"2024-04-01T00:14:44.568759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586470+0000","last_change":"2024-04-01T00:14:44.581854+0000","last_active":"2024-04-01T00:14:44.586470+0000","last_peered":"2024-04-01T00:14:44.586470+0000","last_clean":"2024-04-01T00:14:44.586470+0000","last_became_active":"2024-04-01T00:14:44.581536+0000","last_became_peered":"2024-04-01T00:14:44.581536+0000","last_unstale":"2024-04-01T00:14:44.586470+0000","last_undegraded":"2024-04-01T00:14:44.586470+0000","last_fullsized":"2024-04-01T00:14:44.586470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566765+0000","last_change":"2024-04-01T00:14:44.566765+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566765+0000","last_undegraded":"2024-04-01T00:14:44.566765+0000","last_fullsized":"2024-04-01T00:14:44.566765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582536+0000","last_change":"2024-04-01T00:14:44.578193+0000","last_active":"2024-04-01T00:14:44.582536+0000","last_peered":"2024-04-01T00:14:44.582536+0000","last_clean":"2024-04-01T00:14:44.582536+0000","last_became_active":"2024-04-01T00:14:44.577675+0000","last_became_peered":"2024-04-01T00:14:44.577675+0000","last_unstale":"2024-04-01T00:14:44.582536+0000","last_undegraded":"2024-04-01T00:14:44.582536+0000","last_fullsized":"2024-04-01T00:14:44.582536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564556+0000","last_change":"2024-04-01T00:14:44.564556+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564556+0000","last_undegraded":"2024-04-01T00:14:44.564556+0000","last_fullsized":"2024-04-01T00:14:44.564556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583762+0000","last_change":"2024-04-01T00:14:44.579376+0000","last_active":"2024-04-01T00:14:44.583762+0000","last_peered":"2024-04-01T00:14:44.583762+0000","last_clean":"2024-04-01T00:14:44.583762+0000","last_became_active":"2024-04-01T00:14:44.579048+0000","last_became_peered":"2024-04-01T00:14:44.579048+0000","last_unstale":"2024-04-01T00:14:44.583762+0000","last_undegraded":"2024-04-01T00:14:44.583762+0000","last_fullsized":"2024-04-01T00:14:44.583762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:29:42.707898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563647+0000","last_change":"2024-04-01T00:14:44.563647+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563647+0000","last_undegraded":"2024-04-01T00:14:44.563647+0000","last_fullsized":"2024-04-01T00:14:44.563647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.580968+0000","last_change":"2024-04-01T00:14:44.573445+0000","last_active":"2024-04-01T00:14:44.580968+0000","last_peered":"2024-04-01T00:14:44.580968+0000","last_clean":"2024-04-01T00:14:44.580968+0000","last_became_active":"2024-04-01T00:14:44.573175+0000","last_became_peered":"2024-04-01T00:14:44.573175+0000","last_unstale":"2024-04-01T00:14:44.580968+0000","last_undegraded":"2024-04-01T00:14:44.580968+0000","last_fullsized":"2024-04-01T00:14:44.580968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:50:51.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567155+0000","last_change":"2024-04-01T00:14:44.567155+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567155+0000","last_undegraded":"2024-04-01T00:14:44.567155+0000","last_fullsized":"2024-04-01T00:14:44.567155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578500+0000","last_change":"2024-04-01T00:14:44.573020+0000","last_active":"2024-04-01T00:14:44.578500+0000","last_peered":"2024-04-01T00:14:44.578500+0000","last_clean":"2024-04-01T00:14:44.578500+0000","last_became_active":"2024-04-01T00:14:44.572756+0000","last_became_peered":"2024-04-01T00:14:44.572756+0000","last_unstale":"2024-04-01T00:14:44.578500+0000","last_undegraded":"2024-04-01T00:14:44.578500+0000","last_fullsized":"2024-04-01T00:14:44.578500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570706+0000","last_change":"2024-04-01T00:14:44.570706+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570706+0000","last_undegraded":"2024-04-01T00:14:44.570706+0000","last_fullsized":"2024-04-01T00:14:44.570706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583807+0000","last_change":"2024-04-01T00:14:44.580598+0000","last_active":"2024-04-01T00:14:44.583807+0000","last_peered":"2024-04-01T00:14:44.583807+0000","last_clean":"2024-04-01T00:14:44.583807+0000","last_became_active":"2024-04-01T00:14:44.580245+0000","last_became_peered":"2024-04-01T00:14:44.580245+0000","last_unstale":"2024-04-01T00:14:44.583807+0000","last_undegraded":"2024-04-01T00:14:44.583807+0000","last_fullsized":"2024-04-01T00:14:44.583807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:31.290107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566150+0000","last_change":"2024-04-01T00:14:44.566150+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566150+0000","last_undegraded":"2024-04-01T00:14:44.566150+0000","last_fullsized":"2024-04-01T00:14:44.566150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583792+0000","last_change":"2024-04-01T00:14:44.580512+0000","last_active":"2024-04-01T00:14:44.583792+0000","last_peered":"2024-04-01T00:14:44.583792+0000","last_clean":"2024-04-01T00:14:44.583792+0000","last_became_active":"2024-04-01T00:14:44.580132+0000","last_became_peered":"2024-04-01T00:14:44.580132+0000","last_unstale":"2024-04-01T00:14:44.583792+0000","last_undegraded":"2024-04-01T00:14:44.583792+0000","last_fullsized":"2024-04-01T00:14:44.583792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569699+0000","last_change":"2024-04-01T00:14:44.569699+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569699+0000","last_undegraded":"2024-04-01T00:14:44.569699+0000","last_fullsized":"2024-04-01T00:14:44.569699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.579022+0000","last_change":"2024-04-01T00:14:44.576879+0000","last_active":"2024-04-01T00:14:44.579022+0000","last_peered":"2024-04-01T00:14:44.579022+0000","last_clean":"2024-04-01T00:14:44.579022+0000","last_became_active":"2024-04-01T00:14:44.576567+0000","last_became_peered":"2024-04-01T00:14:44.576567+0000","last_unstale":"2024-04-01T00:14:44.579022+0000","last_undegraded":"2024-04-01T00:14:44.579022+0000","last_fullsized":"2024-04-01T00:14:44.579022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564049+0000","last_change":"2024-04-01T00:14:44.564049+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564049+0000","last_undegraded":"2024-04-01T00:14:44.564049+0000","last_fullsized":"2024-04-01T00:14:44.564049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581075+0000","last_change":"2024-04-01T00:14:44.574281+0000","last_active":"2024-04-01T00:14:44.581075+0000","last_peered":"2024-04-01T00:14:44.581075+0000","last_clean":"2024-04-01T00:14:44.581075+0000","last_became_active":"2024-04-01T00:14:44.573981+0000","last_became_peered":"2024-04-01T00:14:44.573981+0000","last_unstale":"2024-04-01T00:14:44.581075+0000","last_undegraded":"2024-04-01T00:14:44.581075+0000","last_fullsized":"2024-04-01T00:14:44.581075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:23:57.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563883+0000","last_change":"2024-04-01T00:14:44.563883+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563883+0000","last_undegraded":"2024-04-01T00:14:44.563883+0000","last_fullsized":"2024-04-01T00:14:44.563883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581078+0000","last_change":"2024-04-01T00:14:44.574997+0000","last_active":"2024-04-01T00:14:44.581078+0000","last_peered":"2024-04-01T00:14:44.581078+0000","last_clean":"2024-04-01T00:14:44.581078+0000","last_became_active":"2024-04-01T00:14:44.574734+0000","last_became_peered":"2024-04-01T00:14:44.574734+0000","last_unstale":"2024-04-01T00:14:44.581078+0000","last_undegraded":"2024-04-01T00:14:44.581078+0000","last_fullsized":"2024-04-01T00:14:44.581078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:54:03.085286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565020+0000","last_change":"2024-04-01T00:14:44.565020+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565020+0000","last_undegraded":"2024-04-01T00:14:44.565020+0000","last_fullsized":"2024-04-01T00:14:44.565020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581089+0000","last_change":"2024-04-01T00:14:44.576591+0000","last_active":"2024-04-01T00:14:44.581089+0000","last_peered":"2024-04-01T00:14:44.581089+0000","last_clean":"2024-04-01T00:14:44.581089+0000","last_became_active":"2024-04-01T00:14:44.576336+0000","last_became_peered":"2024-04-01T00:14:44.576336+0000","last_unstale":"2024-04-01T00:14:44.581089+0000","last_undegraded":"2024-04-01T00:14:44.581089+0000","last_fullsized":"2024-04-01T00:14:44.581089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:25:30.722477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586233+0000","last_change":"2024-04-01T00:14:44.579815+0000","last_active":"2024-04-01T00:14:44.586233+0000","last_peered":"2024-04-01T00:14:44.586233+0000","last_clean":"2024-04-01T00:14:44.586233+0000","last_became_active":"2024-04-01T00:14:44.579502+0000","last_became_peered":"2024-04-01T00:14:44.579502+0000","last_unstale":"2024-04-01T00:14:44.586233+0000","last_undegraded":"2024-04-01T00:14:44.586233+0000","last_fullsized":"2024-04-01T00:14:44.586233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565505+0000","last_change":"2024-04-01T00:14:44.565505+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565505+0000","last_undegraded":"2024-04-01T00:14:44.565505+0000","last_fullsized":"2024-04-01T00:14:44.565505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585273+0000","last_change":"2024-04-01T00:14:44.579144+0000","last_active":"2024-04-01T00:14:44.585273+0000","last_peered":"2024-04-01T00:14:44.585273+0000","last_clean":"2024-04-01T00:14:44.585273+0000","last_became_active":"2024-04-01T00:14:44.578861+0000","last_became_peered":"2024-04-01T00:14:44.578861+0000","last_unstale":"2024-04-01T00:14:44.585273+0000","last_undegraded":"2024-04-01T00:14:44.585273+0000","last_fullsized":"2024-04-01T00:14:44.585273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564801+0000","last_change":"2024-04-01T00:14:44.564801+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564801+0000","last_undegraded":"2024-04-01T00:14:44.564801+0000","last_fullsized":"2024-04-01T00:14:44.564801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582727+0000","last_change":"2024-04-01T00:14:44.579438+0000","last_active":"2024-04-01T00:14:44.582727+0000","last_peered":"2024-04-01T00:14:44.582727+0000","last_clean":"2024-04-01T00:14:44.582727+0000","last_became_active":"2024-04-01T00:14:44.579181+0000","last_became_peered":"2024-04-01T00:14:44.579181+0000","last_unstale":"2024-04-01T00:14:44.582727+0000","last_undegraded":"2024-04-01T00:14:44.582727+0000","last_fullsized":"2024-04-01T00:14:44.582727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:51:52.461134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569733+0000","last_change":"2024-04-01T00:14:44.569733+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569733+0000","last_undegraded":"2024-04-01T00:14:44.569733+0000","last_fullsized":"2024-04-01T00:14:44.569733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582543+0000","last_change":"2024-04-01T00:14:44.577757+0000","last_active":"2024-04-01T00:14:44.582543+0000","last_peered":"2024-04-01T00:14:44.582543+0000","last_clean":"2024-04-01T00:14:44.582543+0000","last_became_active":"2024-04-01T00:14:44.577431+0000","last_became_peered":"2024-04-01T00:14:44.577431+0000","last_unstale":"2024-04-01T00:14:44.582543+0000","last_undegraded":"2024-04-01T00:14:44.582543+0000","last_fullsized":"2024-04-01T00:14:44.582543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564992+0000","last_change":"2024-04-01T00:14:44.564992+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564992+0000","last_undegraded":"2024-04-01T00:14:44.564992+0000","last_fullsized":"2024-04-01T00:14:44.564992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.568695+0000","last_change":"2024-04-01T00:14:44.567605+0000","last_active":"2024-04-01T00:14:44.568695+0000","last_peered":"2024-04-01T00:14:44.568695+0000","last_clean":"2024-04-01T00:14:44.568695+0000","last_became_active":"2024-04-01T00:14:44.567238+0000","last_became_peered":"2024-04-01T00:14:44.567238+0000","last_unstale":"2024-04-01T00:14:44.568695+0000","last_undegraded":"2024-04-01T00:14:44.568695+0000","last_fullsized":"2024-04-01T00:14:44.568695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563568+0000","last_change":"2024-04-01T00:14:44.563568+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563568+0000","last_undegraded":"2024-04-01T00:14:44.563568+0000","last_fullsized":"2024-04-01T00:14:44.563568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586191+0000","last_change":"2024-04-01T00:14:44.579442+0000","last_active":"2024-04-01T00:14:44.586191+0000","last_peered":"2024-04-01T00:14:44.586191+0000","last_clean":"2024-04-01T00:14:44.586191+0000","last_became_active":"2024-04-01T00:14:44.579151+0000","last_became_peered":"2024-04-01T00:14:44.579151+0000","last_unstale":"2024-04-01T00:14:44.586191+0000","last_undegraded":"2024-04-01T00:14:44.586191+0000","last_fullsized":"2024-04-01T00:14:44.586191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566707+0000","last_change":"2024-04-01T00:14:44.566707+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566707+0000","last_undegraded":"2024-04-01T00:14:44.566707+0000","last_fullsized":"2024-04-01T00:14:44.566707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.576234+0000","last_change":"2024-04-01T00:14:44.570761+0000","last_active":"2024-04-01T00:14:44.576234+0000","last_peered":"2024-04-01T00:14:44.576234+0000","last_clean":"2024-04-01T00:14:44.576234+0000","last_became_active":"2024-04-01T00:14:44.569379+0000","last_became_peered":"2024-04-01T00:14:44.569379+0000","last_unstale":"2024-04-01T00:14:44.576234+0000","last_undegraded":"2024-04-01T00:14:44.576234+0000","last_fullsized":"2024-04-01T00:14:44.576234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:07.835799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566681+0000","last_change":"2024-04-01T00:14:44.566681+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566681+0000","last_undegraded":"2024-04-01T00:14:44.566681+0000","last_fullsized":"2024-04-01T00:14:44.566681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586132+0000","last_change":"2024-04-01T00:14:44.579166+0000","last_active":"2024-04-01T00:14:44.586132+0000","last_peered":"2024-04-01T00:14:44.586132+0000","last_clean":"2024-04-01T00:14:44.586132+0000","last_became_active":"2024-04-01T00:14:44.578892+0000","last_became_peered":"2024-04-01T00:14:44.578892+0000","last_unstale":"2024-04-01T00:14:44.586132+0000","last_undegraded":"2024-04-01T00:14:44.586132+0000","last_fullsized":"2024-04-01T00:14:44.586132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569677+0000","last_change":"2024-04-01T00:14:44.569677+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569677+0000","last_undegraded":"2024-04-01T00:14:44.569677+0000","last_fullsized":"2024-04-01T00:14:44.569677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.573624+0000","last_change":"2024-04-01T00:14:44.568541+0000","last_active":"2024-04-01T00:14:44.573624+0000","last_peered":"2024-04-01T00:14:44.573624+0000","last_clean":"2024-04-01T00:14:44.573624+0000","last_became_active":"2024-04-01T00:14:44.568266+0000","last_became_peered":"2024-04-01T00:14:44.568266+0000","last_unstale":"2024-04-01T00:14:44.573624+0000","last_undegraded":"2024-04-01T00:14:44.573624+0000","last_fullsized":"2024-04-01T00:14:44.573624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567011+0000","last_change":"2024-04-01T00:14:44.567011+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567011+0000","last_undegraded":"2024-04-01T00:14:44.567011+0000","last_fullsized":"2024-04-01T00:14:44.567011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578489+0000","last_change":"2024-04-01T00:14:44.571915+0000","last_active":"2024-04-01T00:14:44.578489+0000","last_peered":"2024-04-01T00:14:44.578489+0000","last_clean":"2024-04-01T00:14:44.578489+0000","last_became_active":"2024-04-01T00:14:44.571316+0000","last_became_peered":"2024-04-01T00:14:44.571316+0000","last_unstale":"2024-04-01T00:14:44.578489+0000","last_undegraded":"2024-04-01T00:14:44.578489+0000","last_fullsized":"2024-04-01T00:14:44.578489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:40:18.733806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565158+0000","last_change":"2024-04-01T00:14:44.565158+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565158+0000","last_undegraded":"2024-04-01T00:14:44.565158+0000","last_fullsized":"2024-04-01T00:14:44.565158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583719+0000","last_change":"2024-04-01T00:14:44.578180+0000","last_active":"2024-04-01T00:14:44.583719+0000","last_peered":"2024-04-01T00:14:44.583719+0000","last_clean":"2024-04-01T00:14:44.583719+0000","last_became_active":"2024-04-01T00:14:44.577563+0000","last_became_peered":"2024-04-01T00:14:44.577563+0000","last_unstale":"2024-04-01T00:14:44.583719+0000","last_undegraded":"2024-04-01T00:14:44.583719+0000","last_fullsized":"2024-04-01T00:14:44.583719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568248+0000","last_change":"2024-04-01T00:14:44.568248+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568248+0000","last_undegraded":"2024-04-01T00:14:44.568248+0000","last_fullsized":"2024-04-01T00:14:44.568248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.587833+0000","last_change":"2024-04-01T00:14:44.581624+0000","last_active":"2024-04-01T00:14:44.587833+0000","last_peered":"2024-04-01T00:14:44.587833+0000","last_clean":"2024-04-01T00:14:44.587833+0000","last_became_active":"2024-04-01T00:14:44.581339+0000","last_became_peered":"2024-04-01T00:14:44.581339+0000","last_unstale":"2024-04-01T00:14:44.587833+0000","last_undegraded":"2024-04-01T00:14:44.587833+0000","last_fullsized":"2024-04-01T00:14:44.587833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.572619+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:14:44.572619+0000","last_peered":"2024-04-01T00:14:44.572619+0000","last_clean":"2024-04-01T00:14:44.572619+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:14:44.572619+0000","last_undegraded":"2024-04-01T00:14:44.572619+0000","last_fullsized":"2024-04-01T00:14:44.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:14:49.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-01T00:14:49.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567529+0000', 'last_change': '2024-04-01T00:14:44.567529+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567529+0000', 'last_undegraded': '2024-04-01T00:14:44.567529+0000', 'last_fullsized': '2024-04-01T00:14:44.567529+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-01T00:14:49.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-01T00:14:49.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568712+0000', 'last_change': '2024-04-01T00:14:44.568712+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568712+0000', 'last_undegraded': '2024-04-01T00:14:44.568712+0000', 'last_fullsized': '2024-04-01T00:14:44.568712+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-01T00:14:49.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563295+0000', 'last_change': '2024-04-01T00:14:44.563295+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563295+0000', 'last_undegraded': '2024-04-01T00:14:44.563295+0000', 'last_fullsized': '2024-04-01T00:14:44.563295+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-01T00:14:49.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564385+0000', 'last_change': '2024-04-01T00:14:44.564385+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564385+0000', 'last_undegraded': '2024-04-01T00:14:44.564385+0000', 'last_fullsized': '2024-04-01T00:14:44.564385+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566470+0000', 'last_change': '2024-04-01T00:14:44.566470+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566470+0000', 'last_undegraded': '2024-04-01T00:14:44.566470+0000', 'last_fullsized': '2024-04-01T00:14:44.566470+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568447+0000', 'last_change': '2024-04-01T00:14:44.568447+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568447+0000', 'last_undegraded': '2024-04-01T00:14:44.568447+0000', 'last_fullsized': '2024-04-01T00:14:44.568447+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-01T00:14:49.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568087+0000', 'last_change': '2024-04-01T00:14:44.568087+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568087+0000', 'last_undegraded': '2024-04-01T00:14:44.568087+0000', 'last_fullsized': '2024-04-01T00:14:44.568087+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-01T00:14:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.571511+0000', 'last_change': '2024-04-01T00:14:44.571511+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.571511+0000', 'last_undegraded': '2024-04-01T00:14:44.571511+0000', 'last_fullsized': '2024-04-01T00:14:44.571511+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-01T00:14:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565973+0000', 'last_change': '2024-04-01T00:14:44.565973+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565973+0000', 'last_undegraded': '2024-04-01T00:14:44.565973+0000', 'last_fullsized': '2024-04-01T00:14:44.565973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-01T00:14:49.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563982+0000', 'last_change': '2024-04-01T00:14:44.563982+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563982+0000', 'last_undegraded': '2024-04-01T00:14:44.563982+0000', 'last_fullsized': '2024-04-01T00:14:44.563982+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-01T00:14:49.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.570322+0000', 'last_change': '2024-04-01T00:14:44.570322+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.570322+0000', 'last_undegraded': '2024-04-01T00:14:44.570322+0000', 'last_fullsized': '2024-04-01T00:14:44.570322+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-01T00:14:49.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563281+0000', 'last_change': '2024-04-01T00:14:44.563281+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563281+0000', 'last_undegraded': '2024-04-01T00:14:44.563281+0000', 'last_fullsized': '2024-04-01T00:14:44.563281+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-01T00:14:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565987+0000', 'last_change': '2024-04-01T00:14:44.565987+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565987+0000', 'last_undegraded': '2024-04-01T00:14:44.565987+0000', 'last_fullsized': '2024-04-01T00:14:44.565987+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-01T00:14:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565613+0000', 'last_change': '2024-04-01T00:14:44.565613+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565613+0000', 'last_undegraded': '2024-04-01T00:14:44.565613+0000', 'last_fullsized': '2024-04-01T00:14:44.565613+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.562449+0000', 'last_change': '2024-04-01T00:14:44.562449+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.562449+0000', 'last_undegraded': '2024-04-01T00:14:44.562449+0000', 'last_fullsized': '2024-04-01T00:14:44.562449+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568007+0000', 'last_change': '2024-04-01T00:14:44.568007+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568007+0000', 'last_undegraded': '2024-04-01T00:14:44.568007+0000', 'last_fullsized': '2024-04-01T00:14:44.568007+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568168+0000', 'last_change': '2024-04-01T00:14:44.568168+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568168+0000', 'last_undegraded': '2024-04-01T00:14:44.568168+0000', 'last_fullsized': '2024-04-01T00:14:44.568168+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-01T00:14:49.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567485+0000', 'last_change': '2024-04-01T00:14:44.567485+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567485+0000', 'last_undegraded': '2024-04-01T00:14:44.567485+0000', 'last_fullsized': '2024-04-01T00:14:44.567485+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-01T00:14:49.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569340+0000', 'last_change': '2024-04-01T00:14:44.569340+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569340+0000', 'last_undegraded': '2024-04-01T00:14:44.569340+0000', 'last_fullsized': '2024-04-01T00:14:44.569340+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-01T00:14:49.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565896+0000', 'last_change': '2024-04-01T00:14:44.565896+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565896+0000', 'last_undegraded': '2024-04-01T00:14:44.565896+0000', 'last_fullsized': '2024-04-01T00:14:44.565896+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-01T00:14:49.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569250+0000', 'last_change': '2024-04-01T00:14:44.569250+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569250+0000', 'last_undegraded': '2024-04-01T00:14:44.569250+0000', 'last_fullsized': '2024-04-01T00:14:44.569250+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-01T00:14:49.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567609+0000', 'last_change': '2024-04-01T00:14:44.567609+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567609+0000', 'last_undegraded': '2024-04-01T00:14:44.567609+0000', 'last_fullsized': '2024-04-01T00:14:44.567609+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-01T00:14:49.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567222+0000', 'last_change': '2024-04-01T00:14:44.567222+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567222+0000', 'last_undegraded': '2024-04-01T00:14:44.567222+0000', 'last_fullsized': '2024-04-01T00:14:44.567222+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-01T00:14:49.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-01T00:14:49.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.562360+0000', 'last_change': '2024-04-01T00:14:44.562360+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.562360+0000', 'last_undegraded': '2024-04-01T00:14:44.562360+0000', 'last_fullsized': '2024-04-01T00:14:44.562360+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-01T00:14:49.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563629+0000', 'last_change': '2024-04-01T00:14:44.563629+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563629+0000', 'last_undegraded': '2024-04-01T00:14:44.563629+0000', 'last_fullsized': '2024-04-01T00:14:44.563629+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566989+0000', 'last_change': '2024-04-01T00:14:44.566989+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566989+0000', 'last_undegraded': '2024-04-01T00:14:44.566989+0000', 'last_fullsized': '2024-04-01T00:14:44.566989+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564557+0000', 'last_change': '2024-04-01T00:14:44.564557+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564557+0000', 'last_undegraded': '2024-04-01T00:14:44.564557+0000', 'last_fullsized': '2024-04-01T00:14:44.564557+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.562736+0000', 'last_change': '2024-04-01T00:14:44.562736+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.562736+0000', 'last_undegraded': '2024-04-01T00:14:44.562736+0000', 'last_fullsized': '2024-04-01T00:14:44.562736+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568778+0000', 'last_change': '2024-04-01T00:14:44.568778+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568778+0000', 'last_undegraded': '2024-04-01T00:14:44.568778+0000', 'last_fullsized': '2024-04-01T00:14:44.568778+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569763+0000', 'last_change': '2024-04-01T00:14:44.569763+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569763+0000', 'last_undegraded': '2024-04-01T00:14:44.569763+0000', 'last_fullsized': '2024-04-01T00:14:44.569763+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567935+0000', 'last_change': '2024-04-01T00:14:44.567935+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567935+0000', 'last_undegraded': '2024-04-01T00:14:44.567935+0000', 'last_fullsized': '2024-04-01T00:14:44.567935+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.571138+0000', 'last_change': '2024-04-01T00:14:44.571138+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.571138+0000', 'last_undegraded': '2024-04-01T00:14:44.571138+0000', 'last_fullsized': '2024-04-01T00:14:44.571138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568613+0000', 'last_change': '2024-04-01T00:14:44.568613+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568613+0000', 'last_undegraded': '2024-04-01T00:14:44.568613+0000', 'last_fullsized': '2024-04-01T00:14:44.568613+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567107+0000', 'last_change': '2024-04-01T00:14:44.567107+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567107+0000', 'last_undegraded': '2024-04-01T00:14:44.567107+0000', 'last_fullsized': '2024-04-01T00:14:44.567107+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-01T00:14:49.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565065+0000', 'last_change': '2024-04-01T00:14:44.565065+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565065+0000', 'last_undegraded': '2024-04-01T00:14:44.565065+0000', 'last_fullsized': '2024-04-01T00:14:44.565065+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563627+0000', 'last_change': '2024-04-01T00:14:44.563627+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563627+0000', 'last_undegraded': '2024-04-01T00:14:44.563627+0000', 'last_fullsized': '2024-04-01T00:14:44.563627+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568087+0000', 'last_change': '2024-04-01T00:14:44.568087+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568087+0000', 'last_undegraded': '2024-04-01T00:14:44.568087+0000', 'last_fullsized': '2024-04-01T00:14:44.568087+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.562887+0000', 'last_change': '2024-04-01T00:14:44.562887+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.562887+0000', 'last_undegraded': '2024-04-01T00:14:44.562887+0000', 'last_fullsized': '2024-04-01T00:14:44.562887+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-01T00:14:49.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567715+0000', 'last_change': '2024-04-01T00:14:44.567715+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567715+0000', 'last_undegraded': '2024-04-01T00:14:44.567715+0000', 'last_fullsized': '2024-04-01T00:14:44.567715+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563749+0000', 'last_change': '2024-04-01T00:14:44.563749+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563749+0000', 'last_undegraded': '2024-04-01T00:14:44.563749+0000', 'last_fullsized': '2024-04-01T00:14:44.563749+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.562810+0000', 'last_change': '2024-04-01T00:14:44.562810+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.562810+0000', 'last_undegraded': '2024-04-01T00:14:44.562810+0000', 'last_fullsized': '2024-04-01T00:14:44.562810+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568759+0000', 'last_change': '2024-04-01T00:14:44.568759+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568759+0000', 'last_undegraded': '2024-04-01T00:14:44.568759+0000', 'last_fullsized': '2024-04-01T00:14:44.568759+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566765+0000', 'last_change': '2024-04-01T00:14:44.566765+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566765+0000', 'last_undegraded': '2024-04-01T00:14:44.566765+0000', 'last_fullsized': '2024-04-01T00:14:44.566765+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564556+0000', 'last_change': '2024-04-01T00:14:44.564556+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564556+0000', 'last_undegraded': '2024-04-01T00:14:44.564556+0000', 'last_fullsized': '2024-04-01T00:14:44.564556+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563647+0000', 'last_change': '2024-04-01T00:14:44.563647+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563647+0000', 'last_undegraded': '2024-04-01T00:14:44.563647+0000', 'last_fullsized': '2024-04-01T00:14:44.563647+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567155+0000', 'last_change': '2024-04-01T00:14:44.567155+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567155+0000', 'last_undegraded': '2024-04-01T00:14:44.567155+0000', 'last_fullsized': '2024-04-01T00:14:44.567155+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.570706+0000', 'last_change': '2024-04-01T00:14:44.570706+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.570706+0000', 'last_undegraded': '2024-04-01T00:14:44.570706+0000', 'last_fullsized': '2024-04-01T00:14:44.570706+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-01T00:14:49.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566150+0000', 'last_change': '2024-04-01T00:14:44.566150+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566150+0000', 'last_undegraded': '2024-04-01T00:14:44.566150+0000', 'last_fullsized': '2024-04-01T00:14:44.566150+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569699+0000', 'last_change': '2024-04-01T00:14:44.569699+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569699+0000', 'last_undegraded': '2024-04-01T00:14:44.569699+0000', 'last_fullsized': '2024-04-01T00:14:44.569699+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564049+0000', 'last_change': '2024-04-01T00:14:44.564049+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564049+0000', 'last_undegraded': '2024-04-01T00:14:44.564049+0000', 'last_fullsized': '2024-04-01T00:14:44.564049+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563883+0000', 'last_change': '2024-04-01T00:14:44.563883+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563883+0000', 'last_undegraded': '2024-04-01T00:14:44.563883+0000', 'last_fullsized': '2024-04-01T00:14:44.563883+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-01T00:14:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565020+0000', 'last_change': '2024-04-01T00:14:44.565020+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565020+0000', 'last_undegraded': '2024-04-01T00:14:44.565020+0000', 'last_fullsized': '2024-04-01T00:14:44.565020+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565973+0000', 'last_change': '2024-04-01T00:14:44.565973+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565973+0000', 'last_undegraded': '2024-04-01T00:14:44.565973+0000', 'last_fullsized': '2024-04-01T00:14:44.565973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565505+0000', 'last_change': '2024-04-01T00:14:44.565505+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565505+0000', 'last_undegraded': '2024-04-01T00:14:44.565505+0000', 'last_fullsized': '2024-04-01T00:14:44.565505+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564801+0000', 'last_change': '2024-04-01T00:14:44.564801+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564801+0000', 'last_undegraded': '2024-04-01T00:14:44.564801+0000', 'last_fullsized': '2024-04-01T00:14:44.564801+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569733+0000', 'last_change': '2024-04-01T00:14:44.569733+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569733+0000', 'last_undegraded': '2024-04-01T00:14:44.569733+0000', 'last_fullsized': '2024-04-01T00:14:44.569733+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-01T00:14:49.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.564992+0000', 'last_change': '2024-04-01T00:14:44.564992+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.564992+0000', 'last_undegraded': '2024-04-01T00:14:44.564992+0000', 'last_fullsized': '2024-04-01T00:14:44.564992+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.563568+0000', 'last_change': '2024-04-01T00:14:44.563568+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.563568+0000', 'last_undegraded': '2024-04-01T00:14:44.563568+0000', 'last_fullsized': '2024-04-01T00:14:44.563568+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566707+0000', 'last_change': '2024-04-01T00:14:44.566707+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566707+0000', 'last_undegraded': '2024-04-01T00:14:44.566707+0000', 'last_fullsized': '2024-04-01T00:14:44.566707+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.566681+0000', 'last_change': '2024-04-01T00:14:44.566681+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.566681+0000', 'last_undegraded': '2024-04-01T00:14:44.566681+0000', 'last_fullsized': '2024-04-01T00:14:44.566681+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-01T00:14:49.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.569677+0000', 'last_change': '2024-04-01T00:14:44.569677+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.569677+0000', 'last_undegraded': '2024-04-01T00:14:44.569677+0000', 'last_fullsized': '2024-04-01T00:14:44.569677+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.567011+0000', 'last_change': '2024-04-01T00:14:44.567011+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.567011+0000', 'last_undegraded': '2024-04-01T00:14:44.567011+0000', 'last_fullsized': '2024-04-01T00:14:44.567011+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.565158+0000', 'last_change': '2024-04-01T00:14:44.565158+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.565158+0000', 'last_undegraded': '2024-04-01T00:14:44.565158+0000', 'last_fullsized': '2024-04-01T00:14:44.565158+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-01T00:14:49.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-01T00:14:44.568248+0000', 'last_change': '2024-04-01T00:14:44.568248+0000', 'last_active': '2024-04-01T00:14:44.553544+0000', 'last_peered': '2024-04-01T00:14:44.553544+0000', 'last_clean': '2024-04-01T00:14:44.553544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-01T00:14:44.568248+0000', 'last_undegraded': '2024-04-01T00:14:44.568248+0000', 'last_fullsized': '2024-04-01T00:14:44.568248+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'last_clean_scrub_stamp': '2024-04-01T00:14:44.553544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-01T00:14:49.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-01T00:14:49.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.418+0000 7ff9cb356640 1 Processor -- start 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.418+0000 7ff9cb356640 1 -- start start 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9cb356640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9c406c3e0 0x7ff9c40d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9cb356640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c40d2780 con 0x7ff9c415e3f0 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9cb356640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c40ce790 con 0x7ff9c40d1920 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9cb356640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7ff9c40cf3a0 con 0x7ff9c406c3e0 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9c98f3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9c406c3e0 0x7ff9c40d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9c88f1640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff9c415e3f0 0x7ff9c40cff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41708/0 (socket says 172.21.15.64:41708) 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.419+0000 7ff9c88f1640 1 -- 172.21.15.64:0/3718811809 learned_addr learned my addr 172.21.15.64:0/3718811809 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c98f3640 1 -- 172.21.15.64:0/3718811809 >> v1:172.21.15.100:6789/0 conn(0x7ff9c40d1920 legacy=0x7ff9c40d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c98f3640 1 -- 172.21.15.64:0/3718811809 >> v1:172.21.15.64:6789/0 conn(0x7ff9c415e3f0 legacy=0x7ff9c40cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c3fff640 1 -- 172.21.15.64:0/3718811809 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365571258 0 0) 0x7ff9c40ce790 con 0x7ff9c40d1920 2024-04-01T00:14:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c98f3640 1 -- 172.21.15.64:0/3718811809 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c40cea70 con 0x7ff9c406c3e0 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c98f3640 1 --2- 172.21.15.64:0/3718811809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c406c3e0 0x7ff9c40d0610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009d90 tx=0x7ff9b402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a395b38ee4d69a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.420+0000 7ff9c3fff640 1 -- 172.21.15.64:0/3718811809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4039070 con 0x7ff9c406c3e0 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9c3fff640 1 -- 172.21.15.64:0/3718811809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b402cab0 con 0x7ff9c406c3e0 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9c3fff640 1 -- 172.21.15.64:0/3718811809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4034440 con 0x7ff9c406c3e0 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 -- 172.21.15.64:0/3718811809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c406c3e0 msgr2=0x7ff9c40d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 --2- 172.21.15.64:0/3718811809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c406c3e0 0x7ff9c40d0610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009d90 tx=0x7ff9b402bd80 comp rx=0 tx=0).stop 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 -- 172.21.15.64:0/3718811809 shutdown_connections 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 --2- 172.21.15.64:0/3718811809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c406c3e0 0x7ff9c40d0610 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 -- 172.21.15.64:0/3718811809 >> 172.21.15.64:0/3718811809 conn(0x7ff9c409a980 msgr2=0x7ff9c409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 -- 172.21.15.64:0/3718811809 shutdown_connections 2024-04-01T00:14:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.421+0000 7ff9cb356640 1 -- 172.21.15.64:0/3718811809 wait complete. 2024-04-01T00:14:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.422+0000 7ff9cb356640 1 Processor -- start 2024-04-01T00:14:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.422+0000 7ff9cb356640 1 -- start start 2024-04-01T00:14:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9c406c3e0 0x7ff9c40b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 0x7ff9c40b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b0cd0 con 0x7ff9c40d1920 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40cf030 con 0x7ff9c41534b0 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9cb356640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c40b0e50 con 0x7ff9c406c3e0 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59774/0 (socket says 172.21.15.64:59774) 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 -- 172.21.15.64:0/2313477239 learned_addr learned my addr 172.21.15.64:0/2313477239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c90f2640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9c406c3e0 0x7ff9c40b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c98f3640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 0x7ff9c40b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 -- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9c406c3e0 msgr2=0x7ff9c40b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9c406c3e0 0x7ff9c40b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 -- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 msgr2=0x7ff9c40b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 0x7ff9c40b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 -- 172.21.15.64:0/2313477239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c4161590 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c98f3640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 0x7ff9c40b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.423+0000 7ff9c88f1640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c620 tx=0x7ff9b800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.424+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8018070 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.424+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b80074e0 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.424+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c40b4f20 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.424+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8007690 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.424+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c40b54e0 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.425+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c40681a0 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.425+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff9b801b070 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.426+0000 7ff9c1ffb640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 0x7ff9b006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.426+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff9b8089be0 con 0x7ff9c40d1920 2024-04-01T00:14:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.426+0000 7ff9c90f2640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 0x7ff9b006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.426+0000 7ff9c90f2640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 0x7ff9b006de50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc001110 tx=0x7ff9bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.430+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff9b80581a0 con 0x7ff9c40d1920 2024-04-01T00:14:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.682+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff9c415d910 con 0x7ff9c40d1920 2024-04-01T00:14:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.682+0000 7ff9c1ffb640 1 -- 172.21.15.64:0/2313477239 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1581 (secure 0 0 0) 0x7ff9b805c240 con 0x7ff9c40d1920 2024-04-01T00:14:49.683 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:49.683 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":87,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1711930413,"num_in_osds":8,"osd_in_since":1711930404,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225247232,"bytes_avail":772868866048,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5224}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-01T00:14:36.754017+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 msgr2=0x7ff9b006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 0x7ff9b006de50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc001110 tx=0x7ff9bc009040 comp rx=0 tx=0).stop 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 msgr2=0x7ff9c4161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c620 tx=0x7ff9b800cae0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 shutdown_connections 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff9b006b990 0x7ff9b006de50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c41534b0 0x7ff9c40b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9c40d1920 0x7ff9c4161050 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 --2- 172.21.15.64:0/2313477239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9c406c3e0 0x7ff9c40b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 >> 172.21.15.64:0/2313477239 conn(0x7ff9c409a980 msgr2=0x7ff9c40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 shutdown_connections 2024-04-01T00:14:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.684+0000 7ff9cb356640 1 -- 172.21.15.64:0/2313477239 wait complete. 2024-04-01T00:14:49.696 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-01T00:14:49.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:14:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.797+0000 7fb2c422a640 1 Processor -- start 2024-04-01T00:14:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.797+0000 7fb2c422a640 1 -- start start 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c422a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2bc0b4350 0x7fb2bc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c422a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 0x7fb2bc06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c422a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2bc0b5d20 con 0x7fb2bc147970 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c422a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc161e10 con 0x7fb2bc0b4350 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c422a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc1620b0 con 0x7fb2bc147420 2024-04-01T00:14:49.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c27c7640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb2bc147970 0x7fb2bc071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41718/0 (socket says 172.21.15.64:41718) 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c27c7640 1 -- 172.21.15.64:0/343451135 learned_addr learned my addr 172.21.15.64:0/343451135 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c1fc6640 1 --2- 172.21.15.64:0/343451135 >> v2:172.21.15.100:3300/0 conn(0x7fb2bc0b4350 0x7fb2bc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c17c5640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 0x7fb2bc06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c0fc4640 1 -- 172.21.15.64:0/343451135 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976922058 0 0) 0x7fb2bc0b5d20 con 0x7fb2bc147970 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c0fc4640 1 -- 172.21.15.64:0/343451135 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2a8003520 con 0x7fb2bc147970 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c1fc6640 1 -- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 msgr2=0x7fb2bc06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c1fc6640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 0x7fb2bc06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c1fc6640 1 -- 172.21.15.64:0/343451135 >> v1:172.21.15.64:6789/0 conn(0x7fb2bc147970 legacy=0x7fb2bc071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c1fc6640 1 -- 172.21.15.64:0/343451135 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc162390 con 0x7fb2bc0b4350 2024-04-01T00:14:49.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.798+0000 7fb2c17c5640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 0x7fb2bc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:49.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.799+0000 7fb2c1fc6640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc0b4350 0x7fb2bc06b0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80097a0 tx=0x7fb2b802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209ad2a12689ea48 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.799+0000 7fb2c0fc4640 1 -- 172.21.15.64:0/343451135 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8035020 con 0x7fb2bc0b4350 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.799+0000 7fb2c0fc4640 1 -- 172.21.15.64:0/343451135 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b8031830 con 0x7fb2bc0b4350 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.799+0000 7fb2c0fc4640 1 -- 172.21.15.64:0/343451135 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8031c00 con 0x7fb2bc0b4350 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 -- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc0b4350 msgr2=0x7fb2bc06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc0b4350 0x7fb2bc06b0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80097a0 tx=0x7fb2b802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 -- 172.21.15.64:0/343451135 shutdown_connections 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc147420 0x7fb2bc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 --2- 172.21.15.64:0/343451135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc0b4350 0x7fb2bc06b0c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 -- 172.21.15.64:0/343451135 >> 172.21.15.64:0/343451135 conn(0x7fb2bc0b2760 msgr2=0x7fb2bc0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 -- 172.21.15.64:0/343451135 shutdown_connections 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.800+0000 7fb2c422a640 1 -- 172.21.15.64:0/343451135 wait complete. 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.801+0000 7fb2c422a640 1 Processor -- start 2024-04-01T00:14:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.801+0000 7fb2c422a640 1 -- start start 2024-04-01T00:14:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 0x7fb2bc1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc147970 0x7fb2bc0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc06ee70 con 0x7fb2bc147420 2024-04-01T00:14:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc162950 con 0x7fb2bc147970 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc06eff0 con 0x7fb2bc0b4350 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c1fc6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 0x7fb2bc1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59790/0 (socket says 172.21.15.64:59790) 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c1fc6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 0x7fb2bc1609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:32830/0 (socket says 172.21.15.64:32830) 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 -- 172.21.15.64:0/1545065002 learned_addr learned my addr 172.21.15.64:0/1545065002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c27c7640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc147970 0x7fb2bc0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 -- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 msgr2=0x7fb2bc1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 0x7fb2bc1609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 -- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc147970 msgr2=0x7fb2bc0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc147970 0x7fb2bc0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 -- 172.21.15.64:0/1545065002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc0c67a0 con 0x7fb2bc147420 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c17c5640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00ec10 tx=0x7fb2ac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac017070 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac00cc30 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc0c6a80 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac013470 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.802+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc0c7040 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.803+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc0681a0 con 0x7fb2bc147420 2024-04-01T00:14:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.804+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb2ac01a070 con 0x7fb2bc147420 2024-04-01T00:14:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.804+0000 7fb2b27fc640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 0x7fb2a006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.804+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb2ac08aa40 con 0x7fb2bc147420 2024-04-01T00:14:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.804+0000 7fb2c1fc6640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 0x7fb2a006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.805+0000 7fb2c1fc6640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 0x7fb2a006de50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8036000 tx=0x7fb2b8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:49.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.809+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb2ac059010 con 0x7fb2bc147420 2024-04-01T00:14:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.987+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2bc0b05c0 con 0x7fb2a006b990 2024-04-01T00:14:49.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.994+0000 7fb2b27fc640 1 -- 172.21.15.64:0/1545065002 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7fb2b8086380 con 0x7fb2a006b990 2024-04-01T00:14:49.997 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:14:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 msgr2=0x7fb2a006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 0x7fb2a006de50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8036000 tx=0x7fb2b8008030 comp rx=0 tx=0).stop 2024-04-01T00:14:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 msgr2=0x7fb2bc1f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00ec10 tx=0x7fb2ac00c520 comp rx=0 tx=0).stop 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 shutdown_connections 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb2a006b990 0x7fb2a006de50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2bc147970 0x7fb2bc0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc147420 0x7fb2bc1f4d60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 --2- 172.21.15.64:0/1545065002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2bc0b4350 0x7fb2bc1609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 >> 172.21.15.64:0/1545065002 conn(0x7fb2bc0b2760 msgr2=0x7fb2bc0b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 shutdown_connections 2024-04-01T00:14:50.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:49.996+0000 7fb2c422a640 1 -- 172.21.15.64:0/1545065002 wait complete. 2024-04-01T00:14:50.008 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-01T00:14:48.756037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219968,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754752,"statfs":{"total":773094113280,"available":772868866048,"internally_reserved":0,"allocated":3899392,"data_stored":2211398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200841"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567529+0000","last_change":"2024-04-01T00:14:44.567529+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567529+0000","last_undegraded":"2024-04-01T00:14:44.567529+0000","last_fullsized":"2024-04-01T00:14:44.567529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568712+0000","last_change":"2024-04-01T00:14:44.568712+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568712+0000","last_undegraded":"2024-04-01T00:14:44.568712+0000","last_fullsized":"2024-04-01T00:14:44.568712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563295+0000","last_change":"2024-04-01T00:14:44.563295+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563295+0000","last_undegraded":"2024-04-01T00:14:44.563295+0000","last_fullsized":"2024-04-01T00:14:44.563295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564385+0000","last_change":"2024-04-01T00:14:44.564385+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564385+0000","last_undegraded":"2024-04-01T00:14:44.564385+0000","last_fullsized":"2024-04-01T00:14:44.564385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566470+0000","last_change":"2024-04-01T00:14:44.566470+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566470+0000","last_undegraded":"2024-04-01T00:14:44.566470+0000","last_fullsized":"2024-04-01T00:14:44.566470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568447+0000","last_change":"2024-04-01T00:14:44.568447+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568447+0000","last_undegraded":"2024-04-01T00:14:44.568447+0000","last_fullsized":"2024-04-01T00:14:44.568447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571511+0000","last_change":"2024-04-01T00:14:44.571511+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571511+0000","last_undegraded":"2024-04-01T00:14:44.571511+0000","last_fullsized":"2024-04-01T00:14:44.571511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563982+0000","last_change":"2024-04-01T00:14:44.563982+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563982+0000","last_undegraded":"2024-04-01T00:14:44.563982+0000","last_fullsized":"2024-04-01T00:14:44.563982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570322+0000","last_change":"2024-04-01T00:14:44.570322+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570322+0000","last_undegraded":"2024-04-01T00:14:44.570322+0000","last_fullsized":"2024-04-01T00:14:44.570322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563281+0000","last_change":"2024-04-01T00:14:44.563281+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563281+0000","last_undegraded":"2024-04-01T00:14:44.563281+0000","last_fullsized":"2024-04-01T00:14:44.563281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565987+0000","last_change":"2024-04-01T00:14:44.565987+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565987+0000","last_undegraded":"2024-04-01T00:14:44.565987+0000","last_fullsized":"2024-04-01T00:14:44.565987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565613+0000","last_change":"2024-04-01T00:14:44.565613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565613+0000","last_undegraded":"2024-04-01T00:14:44.565613+0000","last_fullsized":"2024-04-01T00:14:44.565613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562449+0000","last_change":"2024-04-01T00:14:44.562449+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562449+0000","last_undegraded":"2024-04-01T00:14:44.562449+0000","last_fullsized":"2024-04-01T00:14:44.562449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568007+0000","last_change":"2024-04-01T00:14:44.568007+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568007+0000","last_undegraded":"2024-04-01T00:14:44.568007+0000","last_fullsized":"2024-04-01T00:14:44.568007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568168+0000","last_change":"2024-04-01T00:14:44.568168+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568168+0000","last_undegraded":"2024-04-01T00:14:44.568168+0000","last_fullsized":"2024-04-01T00:14:44.568168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567485+0000","last_change":"2024-04-01T00:14:44.567485+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567485+0000","last_undegraded":"2024-04-01T00:14:44.567485+0000","last_fullsized":"2024-04-01T00:14:44.567485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569340+0000","last_change":"2024-04-01T00:14:44.569340+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569340+0000","last_undegraded":"2024-04-01T00:14:44.569340+0000","last_fullsized":"2024-04-01T00:14:44.569340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565896+0000","last_change":"2024-04-01T00:14:44.565896+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565896+0000","last_undegraded":"2024-04-01T00:14:44.565896+0000","last_fullsized":"2024-04-01T00:14:44.565896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569250+0000","last_change":"2024-04-01T00:14:44.569250+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569250+0000","last_undegraded":"2024-04-01T00:14:44.569250+0000","last_fullsized":"2024-04-01T00:14:44.569250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567609+0000","last_change":"2024-04-01T00:14:44.567609+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567609+0000","last_undegraded":"2024-04-01T00:14:44.567609+0000","last_fullsized":"2024-04-01T00:14:44.567609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567222+0000","last_change":"2024-04-01T00:14:44.567222+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567222+0000","last_undegraded":"2024-04-01T00:14:44.567222+0000","last_fullsized":"2024-04-01T00:14:44.567222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562360+0000","last_change":"2024-04-01T00:14:44.562360+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562360+0000","last_undegraded":"2024-04-01T00:14:44.562360+0000","last_fullsized":"2024-04-01T00:14:44.562360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563629+0000","last_change":"2024-04-01T00:14:44.563629+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563629+0000","last_undegraded":"2024-04-01T00:14:44.563629+0000","last_fullsized":"2024-04-01T00:14:44.563629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566989+0000","last_change":"2024-04-01T00:14:44.566989+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566989+0000","last_undegraded":"2024-04-01T00:14:44.566989+0000","last_fullsized":"2024-04-01T00:14:44.566989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564557+0000","last_change":"2024-04-01T00:14:44.564557+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564557+0000","last_undegraded":"2024-04-01T00:14:44.564557+0000","last_fullsized":"2024-04-01T00:14:44.564557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562736+0000","last_change":"2024-04-01T00:14:44.562736+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562736+0000","last_undegraded":"2024-04-01T00:14:44.562736+0000","last_fullsized":"2024-04-01T00:14:44.562736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568778+0000","last_change":"2024-04-01T00:14:44.568778+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568778+0000","last_undegraded":"2024-04-01T00:14:44.568778+0000","last_fullsized":"2024-04-01T00:14:44.568778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569763+0000","last_change":"2024-04-01T00:14:44.569763+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569763+0000","last_undegraded":"2024-04-01T00:14:44.569763+0000","last_fullsized":"2024-04-01T00:14:44.569763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567935+0000","last_change":"2024-04-01T00:14:44.567935+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567935+0000","last_undegraded":"2024-04-01T00:14:44.567935+0000","last_fullsized":"2024-04-01T00:14:44.567935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.571138+0000","last_change":"2024-04-01T00:14:44.571138+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.571138+0000","last_undegraded":"2024-04-01T00:14:44.571138+0000","last_fullsized":"2024-04-01T00:14:44.571138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568613+0000","last_change":"2024-04-01T00:14:44.568613+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568613+0000","last_undegraded":"2024-04-01T00:14:44.568613+0000","last_fullsized":"2024-04-01T00:14:44.568613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585249+0000","last_change":"2024-04-01T00:14:44.579132+0000","last_active":"2024-04-01T00:14:44.585249+0000","last_peered":"2024-04-01T00:14:44.585249+0000","last_clean":"2024-04-01T00:14:44.585249+0000","last_became_active":"2024-04-01T00:14:44.578796+0000","last_became_peered":"2024-04-01T00:14:44.578796+0000","last_unstale":"2024-04-01T00:14:44.585249+0000","last_undegraded":"2024-04-01T00:14:44.585249+0000","last_fullsized":"2024-04-01T00:14:44.585249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567107+0000","last_change":"2024-04-01T00:14:44.567107+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567107+0000","last_undegraded":"2024-04-01T00:14:44.567107+0000","last_fullsized":"2024-04-01T00:14:44.567107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586107+0000","last_change":"2024-04-01T00:14:44.577986+0000","last_active":"2024-04-01T00:14:44.586107+0000","last_peered":"2024-04-01T00:14:44.586107+0000","last_clean":"2024-04-01T00:14:44.586107+0000","last_became_active":"2024-04-01T00:14:44.577654+0000","last_became_peered":"2024-04-01T00:14:44.577654+0000","last_unstale":"2024-04-01T00:14:44.586107+0000","last_undegraded":"2024-04-01T00:14:44.586107+0000","last_fullsized":"2024-04-01T00:14:44.586107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:27.174967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565065+0000","last_change":"2024-04-01T00:14:44.565065+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565065+0000","last_undegraded":"2024-04-01T00:14:44.565065+0000","last_fullsized":"2024-04-01T00:14:44.565065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583764+0000","last_change":"2024-04-01T00:14:44.579540+0000","last_active":"2024-04-01T00:14:44.583764+0000","last_peered":"2024-04-01T00:14:44.583764+0000","last_clean":"2024-04-01T00:14:44.583764+0000","last_became_active":"2024-04-01T00:14:44.579244+0000","last_became_peered":"2024-04-01T00:14:44.579244+0000","last_unstale":"2024-04-01T00:14:44.583764+0000","last_undegraded":"2024-04-01T00:14:44.583764+0000","last_fullsized":"2024-04-01T00:14:44.583764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563627+0000","last_change":"2024-04-01T00:14:44.563627+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563627+0000","last_undegraded":"2024-04-01T00:14:44.563627+0000","last_fullsized":"2024-04-01T00:14:44.563627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.575359+0000","last_change":"2024-04-01T00:14:44.571485+0000","last_active":"2024-04-01T00:14:44.575359+0000","last_peered":"2024-04-01T00:14:44.575359+0000","last_clean":"2024-04-01T00:14:44.575359+0000","last_became_active":"2024-04-01T00:14:44.571204+0000","last_became_peered":"2024-04-01T00:14:44.571204+0000","last_unstale":"2024-04-01T00:14:44.575359+0000","last_undegraded":"2024-04-01T00:14:44.575359+0000","last_fullsized":"2024-04-01T00:14:44.575359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568087+0000","last_change":"2024-04-01T00:14:44.568087+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568087+0000","last_undegraded":"2024-04-01T00:14:44.568087+0000","last_fullsized":"2024-04-01T00:14:44.568087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585527+0000","last_change":"2024-04-01T00:14:44.581975+0000","last_active":"2024-04-01T00:14:44.585527+0000","last_peered":"2024-04-01T00:14:44.585527+0000","last_clean":"2024-04-01T00:14:44.585527+0000","last_became_active":"2024-04-01T00:14:44.581662+0000","last_became_peered":"2024-04-01T00:14:44.581662+0000","last_unstale":"2024-04-01T00:14:44.585527+0000","last_undegraded":"2024-04-01T00:14:44.585527+0000","last_fullsized":"2024-04-01T00:14:44.585527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:13:03.945693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562887+0000","last_change":"2024-04-01T00:14:44.562887+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562887+0000","last_undegraded":"2024-04-01T00:14:44.562887+0000","last_fullsized":"2024-04-01T00:14:44.562887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581059+0000","last_change":"2024-04-01T00:14:44.574617+0000","last_active":"2024-04-01T00:14:44.581059+0000","last_peered":"2024-04-01T00:14:44.581059+0000","last_clean":"2024-04-01T00:14:44.581059+0000","last_became_active":"2024-04-01T00:14:44.574266+0000","last_became_peered":"2024-04-01T00:14:44.574266+0000","last_unstale":"2024-04-01T00:14:44.581059+0000","last_undegraded":"2024-04-01T00:14:44.581059+0000","last_fullsized":"2024-04-01T00:14:44.581059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:39:36.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567715+0000","last_change":"2024-04-01T00:14:44.567715+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567715+0000","last_undegraded":"2024-04-01T00:14:44.567715+0000","last_fullsized":"2024-04-01T00:14:44.567715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583636+0000","last_change":"2024-04-01T00:14:44.577049+0000","last_active":"2024-04-01T00:14:44.583636+0000","last_peered":"2024-04-01T00:14:44.583636+0000","last_clean":"2024-04-01T00:14:44.583636+0000","last_became_active":"2024-04-01T00:14:44.576762+0000","last_became_peered":"2024-04-01T00:14:44.576762+0000","last_unstale":"2024-04-01T00:14:44.583636+0000","last_undegraded":"2024-04-01T00:14:44.583636+0000","last_fullsized":"2024-04-01T00:14:44.583636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563749+0000","last_change":"2024-04-01T00:14:44.563749+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563749+0000","last_undegraded":"2024-04-01T00:14:44.563749+0000","last_fullsized":"2024-04-01T00:14:44.563749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582683+0000","last_change":"2024-04-01T00:14:44.578322+0000","last_active":"2024-04-01T00:14:44.582683+0000","last_peered":"2024-04-01T00:14:44.582683+0000","last_clean":"2024-04-01T00:14:44.582683+0000","last_became_active":"2024-04-01T00:14:44.578036+0000","last_became_peered":"2024-04-01T00:14:44.578036+0000","last_unstale":"2024-04-01T00:14:44.582683+0000","last_undegraded":"2024-04-01T00:14:44.582683+0000","last_fullsized":"2024-04-01T00:14:44.582683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:10:39.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.562810+0000","last_change":"2024-04-01T00:14:44.562810+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.562810+0000","last_undegraded":"2024-04-01T00:14:44.562810+0000","last_fullsized":"2024-04-01T00:14:44.562810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586393+0000","last_change":"2024-04-01T00:14:44.581137+0000","last_active":"2024-04-01T00:14:44.586393+0000","last_peered":"2024-04-01T00:14:44.586393+0000","last_clean":"2024-04-01T00:14:44.586393+0000","last_became_active":"2024-04-01T00:14:44.580866+0000","last_became_peered":"2024-04-01T00:14:44.580866+0000","last_unstale":"2024-04-01T00:14:44.586393+0000","last_undegraded":"2024-04-01T00:14:44.586393+0000","last_fullsized":"2024-04-01T00:14:44.586393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:03.887331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568759+0000","last_change":"2024-04-01T00:14:44.568759+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568759+0000","last_undegraded":"2024-04-01T00:14:44.568759+0000","last_fullsized":"2024-04-01T00:14:44.568759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586470+0000","last_change":"2024-04-01T00:14:44.581854+0000","last_active":"2024-04-01T00:14:44.586470+0000","last_peered":"2024-04-01T00:14:44.586470+0000","last_clean":"2024-04-01T00:14:44.586470+0000","last_became_active":"2024-04-01T00:14:44.581536+0000","last_became_peered":"2024-04-01T00:14:44.581536+0000","last_unstale":"2024-04-01T00:14:44.586470+0000","last_undegraded":"2024-04-01T00:14:44.586470+0000","last_fullsized":"2024-04-01T00:14:44.586470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566765+0000","last_change":"2024-04-01T00:14:44.566765+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566765+0000","last_undegraded":"2024-04-01T00:14:44.566765+0000","last_fullsized":"2024-04-01T00:14:44.566765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582536+0000","last_change":"2024-04-01T00:14:44.578193+0000","last_active":"2024-04-01T00:14:44.582536+0000","last_peered":"2024-04-01T00:14:44.582536+0000","last_clean":"2024-04-01T00:14:44.582536+0000","last_became_active":"2024-04-01T00:14:44.577675+0000","last_became_peered":"2024-04-01T00:14:44.577675+0000","last_unstale":"2024-04-01T00:14:44.582536+0000","last_undegraded":"2024-04-01T00:14:44.582536+0000","last_fullsized":"2024-04-01T00:14:44.582536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564556+0000","last_change":"2024-04-01T00:14:44.564556+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564556+0000","last_undegraded":"2024-04-01T00:14:44.564556+0000","last_fullsized":"2024-04-01T00:14:44.564556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583762+0000","last_change":"2024-04-01T00:14:44.579376+0000","last_active":"2024-04-01T00:14:44.583762+0000","last_peered":"2024-04-01T00:14:44.583762+0000","last_clean":"2024-04-01T00:14:44.583762+0000","last_became_active":"2024-04-01T00:14:44.579048+0000","last_became_peered":"2024-04-01T00:14:44.579048+0000","last_unstale":"2024-04-01T00:14:44.583762+0000","last_undegraded":"2024-04-01T00:14:44.583762+0000","last_fullsized":"2024-04-01T00:14:44.583762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:29:42.707898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563647+0000","last_change":"2024-04-01T00:14:44.563647+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563647+0000","last_undegraded":"2024-04-01T00:14:44.563647+0000","last_fullsized":"2024-04-01T00:14:44.563647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.580968+0000","last_change":"2024-04-01T00:14:44.573445+0000","last_active":"2024-04-01T00:14:44.580968+0000","last_peered":"2024-04-01T00:14:44.580968+0000","last_clean":"2024-04-01T00:14:44.580968+0000","last_became_active":"2024-04-01T00:14:44.573175+0000","last_became_peered":"2024-04-01T00:14:44.573175+0000","last_unstale":"2024-04-01T00:14:44.580968+0000","last_undegraded":"2024-04-01T00:14:44.580968+0000","last_fullsized":"2024-04-01T00:14:44.580968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:50:51.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567155+0000","last_change":"2024-04-01T00:14:44.567155+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567155+0000","last_undegraded":"2024-04-01T00:14:44.567155+0000","last_fullsized":"2024-04-01T00:14:44.567155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578500+0000","last_change":"2024-04-01T00:14:44.573020+0000","last_active":"2024-04-01T00:14:44.578500+0000","last_peered":"2024-04-01T00:14:44.578500+0000","last_clean":"2024-04-01T00:14:44.578500+0000","last_became_active":"2024-04-01T00:14:44.572756+0000","last_became_peered":"2024-04-01T00:14:44.572756+0000","last_unstale":"2024-04-01T00:14:44.578500+0000","last_undegraded":"2024-04-01T00:14:44.578500+0000","last_fullsized":"2024-04-01T00:14:44.578500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.570706+0000","last_change":"2024-04-01T00:14:44.570706+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.570706+0000","last_undegraded":"2024-04-01T00:14:44.570706+0000","last_fullsized":"2024-04-01T00:14:44.570706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583807+0000","last_change":"2024-04-01T00:14:44.580598+0000","last_active":"2024-04-01T00:14:44.583807+0000","last_peered":"2024-04-01T00:14:44.583807+0000","last_clean":"2024-04-01T00:14:44.583807+0000","last_became_active":"2024-04-01T00:14:44.580245+0000","last_became_peered":"2024-04-01T00:14:44.580245+0000","last_unstale":"2024-04-01T00:14:44.583807+0000","last_undegraded":"2024-04-01T00:14:44.583807+0000","last_fullsized":"2024-04-01T00:14:44.583807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:31.290107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566150+0000","last_change":"2024-04-01T00:14:44.566150+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566150+0000","last_undegraded":"2024-04-01T00:14:44.566150+0000","last_fullsized":"2024-04-01T00:14:44.566150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583792+0000","last_change":"2024-04-01T00:14:44.580512+0000","last_active":"2024-04-01T00:14:44.583792+0000","last_peered":"2024-04-01T00:14:44.583792+0000","last_clean":"2024-04-01T00:14:44.583792+0000","last_became_active":"2024-04-01T00:14:44.580132+0000","last_became_peered":"2024-04-01T00:14:44.580132+0000","last_unstale":"2024-04-01T00:14:44.583792+0000","last_undegraded":"2024-04-01T00:14:44.583792+0000","last_fullsized":"2024-04-01T00:14:44.583792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:24.980730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569699+0000","last_change":"2024-04-01T00:14:44.569699+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569699+0000","last_undegraded":"2024-04-01T00:14:44.569699+0000","last_fullsized":"2024-04-01T00:14:44.569699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.579022+0000","last_change":"2024-04-01T00:14:44.576879+0000","last_active":"2024-04-01T00:14:44.579022+0000","last_peered":"2024-04-01T00:14:44.579022+0000","last_clean":"2024-04-01T00:14:44.579022+0000","last_became_active":"2024-04-01T00:14:44.576567+0000","last_became_peered":"2024-04-01T00:14:44.576567+0000","last_unstale":"2024-04-01T00:14:44.579022+0000","last_undegraded":"2024-04-01T00:14:44.579022+0000","last_fullsized":"2024-04-01T00:14:44.579022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564049+0000","last_change":"2024-04-01T00:14:44.564049+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564049+0000","last_undegraded":"2024-04-01T00:14:44.564049+0000","last_fullsized":"2024-04-01T00:14:44.564049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581075+0000","last_change":"2024-04-01T00:14:44.574281+0000","last_active":"2024-04-01T00:14:44.581075+0000","last_peered":"2024-04-01T00:14:44.581075+0000","last_clean":"2024-04-01T00:14:44.581075+0000","last_became_active":"2024-04-01T00:14:44.573981+0000","last_became_peered":"2024-04-01T00:14:44.573981+0000","last_unstale":"2024-04-01T00:14:44.581075+0000","last_undegraded":"2024-04-01T00:14:44.581075+0000","last_fullsized":"2024-04-01T00:14:44.581075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:23:57.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563883+0000","last_change":"2024-04-01T00:14:44.563883+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563883+0000","last_undegraded":"2024-04-01T00:14:44.563883+0000","last_fullsized":"2024-04-01T00:14:44.563883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581078+0000","last_change":"2024-04-01T00:14:44.574997+0000","last_active":"2024-04-01T00:14:44.581078+0000","last_peered":"2024-04-01T00:14:44.581078+0000","last_clean":"2024-04-01T00:14:44.581078+0000","last_became_active":"2024-04-01T00:14:44.574734+0000","last_became_peered":"2024-04-01T00:14:44.574734+0000","last_unstale":"2024-04-01T00:14:44.581078+0000","last_undegraded":"2024-04-01T00:14:44.581078+0000","last_fullsized":"2024-04-01T00:14:44.581078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:54:03.085286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565020+0000","last_change":"2024-04-01T00:14:44.565020+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565020+0000","last_undegraded":"2024-04-01T00:14:44.565020+0000","last_fullsized":"2024-04-01T00:14:44.565020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.581089+0000","last_change":"2024-04-01T00:14:44.576591+0000","last_active":"2024-04-01T00:14:44.581089+0000","last_peered":"2024-04-01T00:14:44.581089+0000","last_clean":"2024-04-01T00:14:44.581089+0000","last_became_active":"2024-04-01T00:14:44.576336+0000","last_became_peered":"2024-04-01T00:14:44.576336+0000","last_unstale":"2024-04-01T00:14:44.581089+0000","last_undegraded":"2024-04-01T00:14:44.581089+0000","last_fullsized":"2024-04-01T00:14:44.581089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:25:30.722477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565973+0000","last_change":"2024-04-01T00:14:44.565973+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565973+0000","last_undegraded":"2024-04-01T00:14:44.565973+0000","last_fullsized":"2024-04-01T00:14:44.565973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586233+0000","last_change":"2024-04-01T00:14:44.579815+0000","last_active":"2024-04-01T00:14:44.586233+0000","last_peered":"2024-04-01T00:14:44.586233+0000","last_clean":"2024-04-01T00:14:44.586233+0000","last_became_active":"2024-04-01T00:14:44.579502+0000","last_became_peered":"2024-04-01T00:14:44.579502+0000","last_unstale":"2024-04-01T00:14:44.586233+0000","last_undegraded":"2024-04-01T00:14:44.586233+0000","last_fullsized":"2024-04-01T00:14:44.586233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565505+0000","last_change":"2024-04-01T00:14:44.565505+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565505+0000","last_undegraded":"2024-04-01T00:14:44.565505+0000","last_fullsized":"2024-04-01T00:14:44.565505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.585273+0000","last_change":"2024-04-01T00:14:44.579144+0000","last_active":"2024-04-01T00:14:44.585273+0000","last_peered":"2024-04-01T00:14:44.585273+0000","last_clean":"2024-04-01T00:14:44.585273+0000","last_became_active":"2024-04-01T00:14:44.578861+0000","last_became_peered":"2024-04-01T00:14:44.578861+0000","last_unstale":"2024-04-01T00:14:44.585273+0000","last_undegraded":"2024-04-01T00:14:44.585273+0000","last_fullsized":"2024-04-01T00:14:44.585273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564801+0000","last_change":"2024-04-01T00:14:44.564801+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564801+0000","last_undegraded":"2024-04-01T00:14:44.564801+0000","last_fullsized":"2024-04-01T00:14:44.564801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582727+0000","last_change":"2024-04-01T00:14:44.579438+0000","last_active":"2024-04-01T00:14:44.582727+0000","last_peered":"2024-04-01T00:14:44.582727+0000","last_clean":"2024-04-01T00:14:44.582727+0000","last_became_active":"2024-04-01T00:14:44.579181+0000","last_became_peered":"2024-04-01T00:14:44.579181+0000","last_unstale":"2024-04-01T00:14:44.582727+0000","last_undegraded":"2024-04-01T00:14:44.582727+0000","last_fullsized":"2024-04-01T00:14:44.582727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:51:52.461134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569733+0000","last_change":"2024-04-01T00:14:44.569733+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569733+0000","last_undegraded":"2024-04-01T00:14:44.569733+0000","last_fullsized":"2024-04-01T00:14:44.569733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.582543+0000","last_change":"2024-04-01T00:14:44.577757+0000","last_active":"2024-04-01T00:14:44.582543+0000","last_peered":"2024-04-01T00:14:44.582543+0000","last_clean":"2024-04-01T00:14:44.582543+0000","last_became_active":"2024-04-01T00:14:44.577431+0000","last_became_peered":"2024-04-01T00:14:44.577431+0000","last_unstale":"2024-04-01T00:14:44.582543+0000","last_undegraded":"2024-04-01T00:14:44.582543+0000","last_fullsized":"2024-04-01T00:14:44.582543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.564992+0000","last_change":"2024-04-01T00:14:44.564992+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.564992+0000","last_undegraded":"2024-04-01T00:14:44.564992+0000","last_fullsized":"2024-04-01T00:14:44.564992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.568695+0000","last_change":"2024-04-01T00:14:44.567605+0000","last_active":"2024-04-01T00:14:44.568695+0000","last_peered":"2024-04-01T00:14:44.568695+0000","last_clean":"2024-04-01T00:14:44.568695+0000","last_became_active":"2024-04-01T00:14:44.567238+0000","last_became_peered":"2024-04-01T00:14:44.567238+0000","last_unstale":"2024-04-01T00:14:44.568695+0000","last_undegraded":"2024-04-01T00:14:44.568695+0000","last_fullsized":"2024-04-01T00:14:44.568695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.563568+0000","last_change":"2024-04-01T00:14:44.563568+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.563568+0000","last_undegraded":"2024-04-01T00:14:44.563568+0000","last_fullsized":"2024-04-01T00:14:44.563568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586191+0000","last_change":"2024-04-01T00:14:44.579442+0000","last_active":"2024-04-01T00:14:44.586191+0000","last_peered":"2024-04-01T00:14:44.586191+0000","last_clean":"2024-04-01T00:14:44.586191+0000","last_became_active":"2024-04-01T00:14:44.579151+0000","last_became_peered":"2024-04-01T00:14:44.579151+0000","last_unstale":"2024-04-01T00:14:44.586191+0000","last_undegraded":"2024-04-01T00:14:44.586191+0000","last_fullsized":"2024-04-01T00:14:44.586191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566707+0000","last_change":"2024-04-01T00:14:44.566707+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566707+0000","last_undegraded":"2024-04-01T00:14:44.566707+0000","last_fullsized":"2024-04-01T00:14:44.566707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.576234+0000","last_change":"2024-04-01T00:14:44.570761+0000","last_active":"2024-04-01T00:14:44.576234+0000","last_peered":"2024-04-01T00:14:44.576234+0000","last_clean":"2024-04-01T00:14:44.576234+0000","last_became_active":"2024-04-01T00:14:44.569379+0000","last_became_peered":"2024-04-01T00:14:44.569379+0000","last_unstale":"2024-04-01T00:14:44.576234+0000","last_undegraded":"2024-04-01T00:14:44.576234+0000","last_fullsized":"2024-04-01T00:14:44.576234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:07.835799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.566681+0000","last_change":"2024-04-01T00:14:44.566681+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.566681+0000","last_undegraded":"2024-04-01T00:14:44.566681+0000","last_fullsized":"2024-04-01T00:14:44.566681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.586132+0000","last_change":"2024-04-01T00:14:44.579166+0000","last_active":"2024-04-01T00:14:44.586132+0000","last_peered":"2024-04-01T00:14:44.586132+0000","last_clean":"2024-04-01T00:14:44.586132+0000","last_became_active":"2024-04-01T00:14:44.578892+0000","last_became_peered":"2024-04-01T00:14:44.578892+0000","last_unstale":"2024-04-01T00:14:44.586132+0000","last_undegraded":"2024-04-01T00:14:44.586132+0000","last_fullsized":"2024-04-01T00:14:44.586132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.569677+0000","last_change":"2024-04-01T00:14:44.569677+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.569677+0000","last_undegraded":"2024-04-01T00:14:44.569677+0000","last_fullsized":"2024-04-01T00:14:44.569677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.573624+0000","last_change":"2024-04-01T00:14:44.568541+0000","last_active":"2024-04-01T00:14:44.573624+0000","last_peered":"2024-04-01T00:14:44.573624+0000","last_clean":"2024-04-01T00:14:44.573624+0000","last_became_active":"2024-04-01T00:14:44.568266+0000","last_became_peered":"2024-04-01T00:14:44.568266+0000","last_unstale":"2024-04-01T00:14:44.573624+0000","last_undegraded":"2024-04-01T00:14:44.573624+0000","last_fullsized":"2024-04-01T00:14:44.573624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.567011+0000","last_change":"2024-04-01T00:14:44.567011+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.567011+0000","last_undegraded":"2024-04-01T00:14:44.567011+0000","last_fullsized":"2024-04-01T00:14:44.567011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.578489+0000","last_change":"2024-04-01T00:14:44.571915+0000","last_active":"2024-04-01T00:14:44.578489+0000","last_peered":"2024-04-01T00:14:44.578489+0000","last_clean":"2024-04-01T00:14:44.578489+0000","last_became_active":"2024-04-01T00:14:44.571316+0000","last_became_peered":"2024-04-01T00:14:44.571316+0000","last_unstale":"2024-04-01T00:14:44.578489+0000","last_undegraded":"2024-04-01T00:14:44.578489+0000","last_fullsized":"2024-04-01T00:14:44.578489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:40:18.733806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.565158+0000","last_change":"2024-04-01T00:14:44.565158+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.565158+0000","last_undegraded":"2024-04-01T00:14:44.565158+0000","last_fullsized":"2024-04-01T00:14:44.565158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.583719+0000","last_change":"2024-04-01T00:14:44.578180+0000","last_active":"2024-04-01T00:14:44.583719+0000","last_peered":"2024-04-01T00:14:44.583719+0000","last_clean":"2024-04-01T00:14:44.583719+0000","last_became_active":"2024-04-01T00:14:44.577563+0000","last_became_peered":"2024-04-01T00:14:44.577563+0000","last_unstale":"2024-04-01T00:14:44.583719+0000","last_undegraded":"2024-04-01T00:14:44.583719+0000","last_fullsized":"2024-04-01T00:14:44.583719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-01T00:14:44.568248+0000","last_change":"2024-04-01T00:14:44.568248+0000","last_active":"2024-04-01T00:14:44.553544+0000","last_peered":"2024-04-01T00:14:44.553544+0000","last_clean":"2024-04-01T00:14:44.553544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-01T00:14:44.568248+0000","last_undegraded":"2024-04-01T00:14:44.568248+0000","last_fullsized":"2024-04-01T00:14:44.568248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.587833+0000","last_change":"2024-04-01T00:14:44.581624+0000","last_active":"2024-04-01T00:14:44.587833+0000","last_peered":"2024-04-01T00:14:44.587833+0000","last_clean":"2024-04-01T00:14:44.587833+0000","last_became_active":"2024-04-01T00:14:44.581339+0000","last_became_peered":"2024-04-01T00:14:44.581339+0000","last_unstale":"2024-04-01T00:14:44.587833+0000","last_undegraded":"2024-04-01T00:14:44.587833+0000","last_fullsized":"2024-04-01T00:14:44.587833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-01T00:14:44.572619+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:14:44.572619+0000","last_peered":"2024-04-01T00:14:44.572619+0000","last_clean":"2024-04-01T00:14:44.572619+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:14:44.572619+0000","last_undegraded":"2024-04-01T00:14:44.572619+0000","last_fullsized":"2024-04-01T00:14:44.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:14:50.552 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T00:14:50.551+0000 7f257d578640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-01T00:14:53.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T00:14:53.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.126+0000 7f6f20b1e640 1 Processor -- start 2024-04-01T00:14:53.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.126+0000 7f6f20b1e640 1 -- start start 2024-04-01T00:14:53.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.127+0000 7f6f20b1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f1c151fe0 0x7f6f1c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.127+0000 7f6f20b1e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6f1c15f610 0x7f6f1c070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.127+0000 7f6f20b1e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c0711e0 con 0x7f6f1c0cff40 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.127+0000 7f6f20b1e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c15f180 con 0x7f6f1c151fe0 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.127+0000 7f6f20b1e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c071460 con 0x7f6f1c15f610 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f1c151fe0 0x7f6f1c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f1c151fe0 0x7f6f1c15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59806/0 (socket says 172.21.15.64:59806) 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 -- 172.21.15.64:0/472621753 learned_addr learned my addr 172.21.15.64:0/472621753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f19d9b640 1 --2- 172.21.15.64:0/472621753 >> v2:172.21.15.100:3300/0 conn(0x7f6f1c15f610 0x7f6f1c070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 -- 172.21.15.64:0/472621753 >> v2:172.21.15.100:3300/0 conn(0x7f6f1c15f610 msgr2=0x7f6f1c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 --2- 172.21.15.64:0/472621753 >> v2:172.21.15.100:3300/0 conn(0x7f6f1c15f610 0x7f6f1c070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 -- 172.21.15.64:0/472621753 >> v1:172.21.15.100:6789/0 conn(0x7f6f1c0cff40 legacy=0x7f6f1c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.128+0000 7f6f1a59c640 1 -- 172.21.15.64:0/472621753 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c071740 con 0x7f6f1c151fe0 2024-04-01T00:14:53.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f1a59c640 1 --2- 172.21.15.64:0/472621753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c15beb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b5160 tx=0x7f6f1402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a72fa819b71773 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f1959a640 1 -- 172.21.15.64:0/472621753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1403b070 con 0x7f6f1c151fe0 2024-04-01T00:14:53.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f1959a640 1 -- 172.21.15.64:0/472621753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f14036440 con 0x7f6f1c151fe0 2024-04-01T00:14:53.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f1959a640 1 -- 172.21.15.64:0/472621753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14036810 con 0x7f6f1c151fe0 2024-04-01T00:14:53.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 -- 172.21.15.64:0/472621753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 msgr2=0x7f6f1c15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/472621753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c15beb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b5160 tx=0x7f6f1402e170 comp rx=0 tx=0).stop 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 -- 172.21.15.64:0/472621753 shutdown_connections 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/472621753 >> v2:172.21.15.100:3300/0 conn(0x7f6f1c15f610 0x7f6f1c070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/472621753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c15beb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 -- 172.21.15.64:0/472621753 >> 172.21.15.64:0/472621753 conn(0x7f6f1c06b4e0 msgr2=0x7f6f1c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 -- 172.21.15.64:0/472621753 shutdown_connections 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.129+0000 7f6f20b1e640 1 -- 172.21.15.64:0/472621753 wait complete. 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.131+0000 7f6f20b1e640 1 Processor -- start 2024-04-01T00:14:53.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.131+0000 7f6f20b1e640 1 -- start start 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1c0cff40 0x7f6f1c1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c15f610 0x7f6f1c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c0d03a0 con 0x7f6f1c151fe0 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c15ee10 con 0x7f6f1c15f610 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f20b1e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c0d0520 con 0x7f6f1c0cff40 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59814/0 (socket says 172.21.15.64:59814) 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 -- 172.21.15.64:0/771020135 learned_addr learned my addr 172.21.15.64:0/771020135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f1a59c640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1c0cff40 0x7f6f1c1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f1ad9d640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c15f610 0x7f6f1c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 -- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1c0cff40 msgr2=0x7f6f1c1499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1c0cff40 0x7f6f1c1499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.132+0000 7f6f19d9b640 1 -- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c15f610 msgr2=0x7f6f1c06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f19d9b640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c15f610 0x7f6f1c06d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f19d9b640 1 -- 172.21.15.64:0/771020135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0b4c50 con 0x7f6f1c151fe0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f19d9b640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400e840 tx=0x7f6f0400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04017070 con 0x7f6f1c151fe0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0400c940 con 0x7f6f1c151fe0 2024-04-01T00:14:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1e4850 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400cd10 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.133+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c0b4f30 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.134+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c1e4c40 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.134+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f6f0401a070 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.135+0000 7f6f12ffd640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 0x7f6efc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.135+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6f0408a5c0 con 0x7f6f1c151fe0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.135+0000 7f6f1a59c640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 0x7f6efc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.136+0000 7f6f1a59c640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 0x7f6efc06de50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f14038000 tx=0x7f6f14032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.139+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f6f04058c10 con 0x7f6f1c151fe0 2024-04-01T00:14:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.315+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f1c0681a0 con 0x7f6efc06b990 2024-04-01T00:14:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.322+0000 7f6f12ffd640 1 -- 172.21.15.64:0/771020135 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f6f14083840 con 0x7f6efc06b990 2024-04-01T00:14:53.326 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 msgr2=0x7f6efc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 0x7f6efc06de50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f14038000 tx=0x7f6f14032000 comp rx=0 tx=0).stop 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 msgr2=0x7f6f1c0b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400e840 tx=0x7f6f0400ed00 comp rx=0 tx=0).stop 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 shutdown_connections 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f6efc06b990 0x7f6efc06de50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c15f610 0x7f6f1c06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c151fe0 0x7f6f1c0b4240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 --2- 172.21.15.64:0/771020135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1c0cff40 0x7f6f1c1499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 >> 172.21.15.64:0/771020135 conn(0x7f6f1c06b4e0 msgr2=0x7f6f1c15d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 shutdown_connections 2024-04-01T00:14:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.324+0000 7f6f20b1e640 1 -- 172.21.15.64:0/771020135 wait complete. 2024-04-01T00:14:53.337 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-01T00:14:52.756728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220356,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754364,"statfs":{"total":773094113280,"available":772868468736,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001458"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891114+0000","last_change":"2024-04-01T00:14:45.578522+0000","last_active":"2024-04-01T00:14:46.891114+0000","last_peered":"2024-04-01T00:14:46.891114+0000","last_clean":"2024-04-01T00:14:46.891114+0000","last_became_active":"2024-04-01T00:14:45.578212+0000","last_became_peered":"2024-04-01T00:14:45.578212+0000","last_unstale":"2024-04-01T00:14:46.891114+0000","last_undegraded":"2024-04-01T00:14:46.891114+0000","last_fullsized":"2024-04-01T00:14:46.891114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:45:07.480935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891626+0000","last_change":"2024-04-01T00:14:45.578642+0000","last_active":"2024-04-01T00:14:46.891626+0000","last_peered":"2024-04-01T00:14:46.891626+0000","last_clean":"2024-04-01T00:14:46.891626+0000","last_became_active":"2024-04-01T00:14:45.578367+0000","last_became_peered":"2024-04-01T00:14:45.578367+0000","last_unstale":"2024-04-01T00:14:46.891626+0000","last_undegraded":"2024-04-01T00:14:46.891626+0000","last_fullsized":"2024-04-01T00:14:46.891626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:36:51.182101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567530+0000","last_change":"2024-04-01T00:14:45.574669+0000","last_active":"2024-04-01T00:14:46.567530+0000","last_peered":"2024-04-01T00:14:46.567530+0000","last_clean":"2024-04-01T00:14:46.567530+0000","last_became_active":"2024-04-01T00:14:45.574392+0000","last_became_peered":"2024-04-01T00:14:45.574392+0000","last_unstale":"2024-04-01T00:14:46.567530+0000","last_undegraded":"2024-04-01T00:14:46.567530+0000","last_fullsized":"2024-04-01T00:14:46.567530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:04:32.069068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563623+0000","last_change":"2024-04-01T00:14:45.578885+0000","last_active":"2024-04-01T00:14:46.563623+0000","last_peered":"2024-04-01T00:14:46.563623+0000","last_clean":"2024-04-01T00:14:46.563623+0000","last_became_active":"2024-04-01T00:14:45.578605+0000","last_became_peered":"2024-04-01T00:14:45.578605+0000","last_unstale":"2024-04-01T00:14:46.563623+0000","last_undegraded":"2024-04-01T00:14:46.563623+0000","last_fullsized":"2024-04-01T00:14:46.563623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:05:55.878807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890487+0000","last_change":"2024-04-01T00:14:45.573094+0000","last_active":"2024-04-01T00:14:46.890487+0000","last_peered":"2024-04-01T00:14:46.890487+0000","last_clean":"2024-04-01T00:14:46.890487+0000","last_became_active":"2024-04-01T00:14:45.572733+0000","last_became_peered":"2024-04-01T00:14:45.572733+0000","last_unstale":"2024-04-01T00:14:46.890487+0000","last_undegraded":"2024-04-01T00:14:46.890487+0000","last_fullsized":"2024-04-01T00:14:46.890487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:28.180250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890826+0000","last_change":"2024-04-01T00:14:45.568673+0000","last_active":"2024-04-01T00:14:46.890826+0000","last_peered":"2024-04-01T00:14:46.890826+0000","last_clean":"2024-04-01T00:14:46.890826+0000","last_became_active":"2024-04-01T00:14:45.568409+0000","last_became_peered":"2024-04-01T00:14:45.568409+0000","last_unstale":"2024-04-01T00:14:46.890826+0000","last_undegraded":"2024-04-01T00:14:46.890826+0000","last_fullsized":"2024-04-01T00:14:46.890826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:03:08.207324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614460+0000","last_change":"2024-04-01T00:14:45.584429+0000","last_active":"2024-04-01T00:14:46.614460+0000","last_peered":"2024-04-01T00:14:46.614460+0000","last_clean":"2024-04-01T00:14:46.614460+0000","last_became_active":"2024-04-01T00:14:45.583589+0000","last_became_peered":"2024-04-01T00:14:45.583589+0000","last_unstale":"2024-04-01T00:14:46.614460+0000","last_undegraded":"2024-04-01T00:14:46.614460+0000","last_fullsized":"2024-04-01T00:14:46.614460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:59:27.154412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614209+0000","last_change":"2024-04-01T00:14:45.582306+0000","last_active":"2024-04-01T00:14:46.614209+0000","last_peered":"2024-04-01T00:14:46.614209+0000","last_clean":"2024-04-01T00:14:46.614209+0000","last_became_active":"2024-04-01T00:14:45.582024+0000","last_became_peered":"2024-04-01T00:14:45.582024+0000","last_unstale":"2024-04-01T00:14:46.614209+0000","last_undegraded":"2024-04-01T00:14:46.614209+0000","last_fullsized":"2024-04-01T00:14:46.614209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:37:47.730151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891388+0000","last_change":"2024-04-01T00:14:45.586534+0000","last_active":"2024-04-01T00:14:46.891388+0000","last_peered":"2024-04-01T00:14:46.891388+0000","last_clean":"2024-04-01T00:14:46.891388+0000","last_became_active":"2024-04-01T00:14:45.582981+0000","last_became_peered":"2024-04-01T00:14:45.582981+0000","last_unstale":"2024-04-01T00:14:46.891388+0000","last_undegraded":"2024-04-01T00:14:46.891388+0000","last_fullsized":"2024-04-01T00:14:46.891388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:10:00.045974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563796+0000","last_change":"2024-04-01T00:14:45.574725+0000","last_active":"2024-04-01T00:14:46.563796+0000","last_peered":"2024-04-01T00:14:46.563796+0000","last_clean":"2024-04-01T00:14:46.563796+0000","last_became_active":"2024-04-01T00:14:45.574451+0000","last_became_peered":"2024-04-01T00:14:45.574451+0000","last_unstale":"2024-04-01T00:14:46.563796+0000","last_undegraded":"2024-04-01T00:14:46.563796+0000","last_fullsized":"2024-04-01T00:14:46.563796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:17:41.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614502+0000","last_change":"2024-04-01T00:14:45.584678+0000","last_active":"2024-04-01T00:14:46.614502+0000","last_peered":"2024-04-01T00:14:46.614502+0000","last_clean":"2024-04-01T00:14:46.614502+0000","last_became_active":"2024-04-01T00:14:45.583775+0000","last_became_peered":"2024-04-01T00:14:45.583775+0000","last_unstale":"2024-04-01T00:14:46.614502+0000","last_undegraded":"2024-04-01T00:14:46.614502+0000","last_fullsized":"2024-04-01T00:14:46.614502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:05:51.351096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567313+0000","last_change":"2024-04-01T00:14:45.572102+0000","last_active":"2024-04-01T00:14:46.567313+0000","last_peered":"2024-04-01T00:14:46.567313+0000","last_clean":"2024-04-01T00:14:46.567313+0000","last_became_active":"2024-04-01T00:14:45.571814+0000","last_became_peered":"2024-04-01T00:14:45.571814+0000","last_unstale":"2024-04-01T00:14:46.567313+0000","last_undegraded":"2024-04-01T00:14:46.567313+0000","last_fullsized":"2024-04-01T00:14:46.567313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:37:47.730151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890625+0000","last_change":"2024-04-01T00:14:45.574841+0000","last_active":"2024-04-01T00:14:46.890625+0000","last_peered":"2024-04-01T00:14:46.890625+0000","last_clean":"2024-04-01T00:14:46.890625+0000","last_became_active":"2024-04-01T00:14:45.574576+0000","last_became_peered":"2024-04-01T00:14:45.574576+0000","last_unstale":"2024-04-01T00:14:46.890625+0000","last_undegraded":"2024-04-01T00:14:46.890625+0000","last_fullsized":"2024-04-01T00:14:46.890625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:39.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890971+0000","last_change":"2024-04-01T00:14:45.572352+0000","last_active":"2024-04-01T00:14:46.890971+0000","last_peered":"2024-04-01T00:14:46.890971+0000","last_clean":"2024-04-01T00:14:46.890971+0000","last_became_active":"2024-04-01T00:14:45.572019+0000","last_became_peered":"2024-04-01T00:14:45.572019+0000","last_unstale":"2024-04-01T00:14:46.890971+0000","last_undegraded":"2024-04-01T00:14:46.890971+0000","last_fullsized":"2024-04-01T00:14:46.890971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:45:46.245904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567204+0000","last_change":"2024-04-01T00:14:45.579636+0000","last_active":"2024-04-01T00:14:46.567204+0000","last_peered":"2024-04-01T00:14:46.567204+0000","last_clean":"2024-04-01T00:14:46.567204+0000","last_became_active":"2024-04-01T00:14:45.579377+0000","last_became_peered":"2024-04-01T00:14:45.579377+0000","last_unstale":"2024-04-01T00:14:46.567204+0000","last_undegraded":"2024-04-01T00:14:46.567204+0000","last_fullsized":"2024-04-01T00:14:46.567204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:18:00.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614542+0000","last_change":"2024-04-01T00:14:45.585738+0000","last_active":"2024-04-01T00:14:46.614542+0000","last_peered":"2024-04-01T00:14:46.614542+0000","last_clean":"2024-04-01T00:14:46.614542+0000","last_became_active":"2024-04-01T00:14:45.585246+0000","last_became_peered":"2024-04-01T00:14:45.585246+0000","last_unstale":"2024-04-01T00:14:46.614542+0000","last_undegraded":"2024-04-01T00:14:46.614542+0000","last_fullsized":"2024-04-01T00:14:46.614542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:04:32.634685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614611+0000","last_change":"2024-04-01T00:14:45.586455+0000","last_active":"2024-04-01T00:14:46.614611+0000","last_peered":"2024-04-01T00:14:46.614611+0000","last_clean":"2024-04-01T00:14:46.614611+0000","last_became_active":"2024-04-01T00:14:45.586120+0000","last_became_peered":"2024-04-01T00:14:45.586120+0000","last_unstale":"2024-04-01T00:14:46.614611+0000","last_undegraded":"2024-04-01T00:14:46.614611+0000","last_fullsized":"2024-04-01T00:14:46.614611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:21:52.022463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891652+0000","last_change":"2024-04-01T00:14:45.576722+0000","last_active":"2024-04-01T00:14:46.891652+0000","last_peered":"2024-04-01T00:14:46.891652+0000","last_clean":"2024-04-01T00:14:46.891652+0000","last_became_active":"2024-04-01T00:14:45.575871+0000","last_became_peered":"2024-04-01T00:14:45.575871+0000","last_unstale":"2024-04-01T00:14:46.891652+0000","last_undegraded":"2024-04-01T00:14:46.891652+0000","last_fullsized":"2024-04-01T00:14:46.891652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:28:08.610493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891476+0000","last_change":"2024-04-01T00:14:45.575446+0000","last_active":"2024-04-01T00:14:46.891476+0000","last_peered":"2024-04-01T00:14:46.891476+0000","last_clean":"2024-04-01T00:14:46.891476+0000","last_became_active":"2024-04-01T00:14:45.575172+0000","last_became_peered":"2024-04-01T00:14:45.575172+0000","last_unstale":"2024-04-01T00:14:46.891476+0000","last_undegraded":"2024-04-01T00:14:46.891476+0000","last_fullsized":"2024-04-01T00:14:46.891476+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:22:51.614543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890950+0000","last_change":"2024-04-01T00:14:45.576164+0000","last_active":"2024-04-01T00:14:46.890950+0000","last_peered":"2024-04-01T00:14:46.890950+0000","last_clean":"2024-04-01T00:14:46.890950+0000","last_became_active":"2024-04-01T00:14:45.575882+0000","last_became_peered":"2024-04-01T00:14:45.575882+0000","last_unstale":"2024-04-01T00:14:46.890950+0000","last_undegraded":"2024-04-01T00:14:46.890950+0000","last_fullsized":"2024-04-01T00:14:46.890950+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:03.645016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890424+0000","last_change":"2024-04-01T00:14:45.576000+0000","last_active":"2024-04-01T00:14:46.890424+0000","last_peered":"2024-04-01T00:14:46.890424+0000","last_clean":"2024-04-01T00:14:46.890424+0000","last_became_active":"2024-04-01T00:14:45.575704+0000","last_became_peered":"2024-04-01T00:14:45.575704+0000","last_unstale":"2024-04-01T00:14:46.890424+0000","last_undegraded":"2024-04-01T00:14:46.890424+0000","last_fullsized":"2024-04-01T00:14:46.890424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:25.986012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890577+0000","last_change":"2024-04-01T00:14:45.574017+0000","last_active":"2024-04-01T00:14:46.890577+0000","last_peered":"2024-04-01T00:14:46.890577+0000","last_clean":"2024-04-01T00:14:46.890577+0000","last_became_active":"2024-04-01T00:14:45.573715+0000","last_became_peered":"2024-04-01T00:14:45.573715+0000","last_unstale":"2024-04-01T00:14:46.890577+0000","last_undegraded":"2024-04-01T00:14:46.890577+0000","last_fullsized":"2024-04-01T00:14:46.890577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:04.892614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566299+0000","last_change":"2024-04-01T00:14:45.584918+0000","last_active":"2024-04-01T00:14:46.566299+0000","last_peered":"2024-04-01T00:14:46.566299+0000","last_clean":"2024-04-01T00:14:46.566299+0000","last_became_active":"2024-04-01T00:14:45.584591+0000","last_became_peered":"2024-04-01T00:14:45.584591+0000","last_unstale":"2024-04-01T00:14:46.566299+0000","last_undegraded":"2024-04-01T00:14:46.566299+0000","last_fullsized":"2024-04-01T00:14:46.566299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:04.892614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567082+0000","last_change":"2024-04-01T00:14:45.573464+0000","last_active":"2024-04-01T00:14:46.567082+0000","last_peered":"2024-04-01T00:14:46.567082+0000","last_clean":"2024-04-01T00:14:46.567082+0000","last_became_active":"2024-04-01T00:14:45.573188+0000","last_became_peered":"2024-04-01T00:14:45.573188+0000","last_unstale":"2024-04-01T00:14:46.567082+0000","last_undegraded":"2024-04-01T00:14:46.567082+0000","last_fullsized":"2024-04-01T00:14:46.567082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:25:08.051094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563826+0000","last_change":"2024-04-01T00:14:45.588329+0000","last_active":"2024-04-01T00:14:46.563826+0000","last_peered":"2024-04-01T00:14:46.563826+0000","last_clean":"2024-04-01T00:14:46.563826+0000","last_became_active":"2024-04-01T00:14:45.588045+0000","last_became_peered":"2024-04-01T00:14:45.588045+0000","last_unstale":"2024-04-01T00:14:46.563826+0000","last_undegraded":"2024-04-01T00:14:46.563826+0000","last_fullsized":"2024-04-01T00:14:46.563826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:21:19.292246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891707+0000","last_change":"2024-04-01T00:14:45.586471+0000","last_active":"2024-04-01T00:14:46.891707+0000","last_peered":"2024-04-01T00:14:46.891707+0000","last_clean":"2024-04-01T00:14:46.891707+0000","last_became_active":"2024-04-01T00:14:45.586113+0000","last_became_peered":"2024-04-01T00:14:45.586113+0000","last_unstale":"2024-04-01T00:14:46.891707+0000","last_undegraded":"2024-04-01T00:14:46.891707+0000","last_fullsized":"2024-04-01T00:14:46.891707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:56:16.907233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563904+0000","last_change":"2024-04-01T00:14:45.579402+0000","last_active":"2024-04-01T00:14:46.563904+0000","last_peered":"2024-04-01T00:14:46.563904+0000","last_clean":"2024-04-01T00:14:46.563904+0000","last_became_active":"2024-04-01T00:14:45.578316+0000","last_became_peered":"2024-04-01T00:14:45.578316+0000","last_unstale":"2024-04-01T00:14:46.563904+0000","last_undegraded":"2024-04-01T00:14:46.563904+0000","last_fullsized":"2024-04-01T00:14:46.563904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:18:48.442342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567758+0000","last_change":"2024-04-01T00:14:45.573047+0000","last_active":"2024-04-01T00:14:46.567758+0000","last_peered":"2024-04-01T00:14:46.567758+0000","last_clean":"2024-04-01T00:14:46.567758+0000","last_became_active":"2024-04-01T00:14:45.572745+0000","last_became_peered":"2024-04-01T00:14:45.572745+0000","last_unstale":"2024-04-01T00:14:46.567758+0000","last_undegraded":"2024-04-01T00:14:46.567758+0000","last_fullsized":"2024-04-01T00:14:46.567758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:03:08.207324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890571+0000","last_change":"2024-04-01T00:14:45.573996+0000","last_active":"2024-04-01T00:14:46.890571+0000","last_peered":"2024-04-01T00:14:46.890571+0000","last_clean":"2024-04-01T00:14:46.890571+0000","last_became_active":"2024-04-01T00:14:45.573731+0000","last_became_peered":"2024-04-01T00:14:45.573731+0000","last_unstale":"2024-04-01T00:14:46.890571+0000","last_undegraded":"2024-04-01T00:14:46.890571+0000","last_fullsized":"2024-04-01T00:14:46.890571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:32.295390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614305+0000","last_change":"2024-04-01T00:14:45.584575+0000","last_active":"2024-04-01T00:14:46.614305+0000","last_peered":"2024-04-01T00:14:46.614305+0000","last_clean":"2024-04-01T00:14:46.614305+0000","last_became_active":"2024-04-01T00:14:45.584167+0000","last_became_peered":"2024-04-01T00:14:45.584167+0000","last_unstale":"2024-04-01T00:14:46.614305+0000","last_undegraded":"2024-04-01T00:14:46.614305+0000","last_fullsized":"2024-04-01T00:14:46.614305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:04:32.069068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890575+0000","last_change":"2024-04-01T00:14:45.578124+0000","last_active":"2024-04-01T00:14:46.890575+0000","last_peered":"2024-04-01T00:14:46.890575+0000","last_clean":"2024-04-01T00:14:46.890575+0000","last_became_active":"2024-04-01T00:14:45.577781+0000","last_became_peered":"2024-04-01T00:14:45.577781+0000","last_unstale":"2024-04-01T00:14:46.890575+0000","last_undegraded":"2024-04-01T00:14:46.890575+0000","last_fullsized":"2024-04-01T00:14:46.890575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:32.295390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614220+0000","last_change":"2024-04-01T00:14:45.585525+0000","last_active":"2024-04-01T00:14:46.614220+0000","last_peered":"2024-04-01T00:14:46.614220+0000","last_clean":"2024-04-01T00:14:46.614220+0000","last_became_active":"2024-04-01T00:14:45.584932+0000","last_became_peered":"2024-04-01T00:14:45.584932+0000","last_unstale":"2024-04-01T00:14:46.614220+0000","last_undegraded":"2024-04-01T00:14:46.614220+0000","last_fullsized":"2024-04-01T00:14:46.614220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:39:53.898702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566456+0000","last_change":"2024-04-01T00:14:45.585261+0000","last_active":"2024-04-01T00:14:46.566456+0000","last_peered":"2024-04-01T00:14:46.566456+0000","last_clean":"2024-04-01T00:14:46.566456+0000","last_became_active":"2024-04-01T00:14:45.584814+0000","last_became_peered":"2024-04-01T00:14:45.584814+0000","last_unstale":"2024-04-01T00:14:46.566456+0000","last_undegraded":"2024-04-01T00:14:46.566456+0000","last_fullsized":"2024-04-01T00:14:46.566456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:01:12.998908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891254+0000","last_change":"2024-04-01T00:14:44.579132+0000","last_active":"2024-04-01T00:14:46.891254+0000","last_peered":"2024-04-01T00:14:46.891254+0000","last_clean":"2024-04-01T00:14:46.891254+0000","last_became_active":"2024-04-01T00:14:44.578796+0000","last_became_peered":"2024-04-01T00:14:44.578796+0000","last_unstale":"2024-04-01T00:14:46.891254+0000","last_undegraded":"2024-04-01T00:14:46.891254+0000","last_fullsized":"2024-04-01T00:14:46.891254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891283+0000","last_change":"2024-04-01T00:14:45.579527+0000","last_active":"2024-04-01T00:14:46.891283+0000","last_peered":"2024-04-01T00:14:46.891283+0000","last_clean":"2024-04-01T00:14:46.891283+0000","last_became_active":"2024-04-01T00:14:45.579243+0000","last_became_peered":"2024-04-01T00:14:45.579243+0000","last_unstale":"2024-04-01T00:14:46.891283+0000","last_undegraded":"2024-04-01T00:14:46.891283+0000","last_fullsized":"2024-04-01T00:14:46.891283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:01:12.998908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566451+0000","last_change":"2024-04-01T00:14:44.577986+0000","last_active":"2024-04-01T00:14:46.566451+0000","last_peered":"2024-04-01T00:14:46.566451+0000","last_clean":"2024-04-01T00:14:46.566451+0000","last_became_active":"2024-04-01T00:14:44.577654+0000","last_became_peered":"2024-04-01T00:14:44.577654+0000","last_unstale":"2024-04-01T00:14:46.566451+0000","last_undegraded":"2024-04-01T00:14:46.566451+0000","last_fullsized":"2024-04-01T00:14:46.566451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891184+0000","last_change":"2024-04-01T00:14:45.573415+0000","last_active":"2024-04-01T00:14:46.891184+0000","last_peered":"2024-04-01T00:14:46.891184+0000","last_clean":"2024-04-01T00:14:46.891184+0000","last_became_active":"2024-04-01T00:14:45.572787+0000","last_became_peered":"2024-04-01T00:14:45.572787+0000","last_unstale":"2024-04-01T00:14:46.891184+0000","last_undegraded":"2024-04-01T00:14:46.891184+0000","last_fullsized":"2024-04-01T00:14:46.891184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:11.720598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891407+0000","last_change":"2024-04-01T00:14:44.579540+0000","last_active":"2024-04-01T00:14:46.891407+0000","last_peered":"2024-04-01T00:14:46.891407+0000","last_clean":"2024-04-01T00:14:46.891407+0000","last_became_active":"2024-04-01T00:14:44.579244+0000","last_became_peered":"2024-04-01T00:14:44.579244+0000","last_unstale":"2024-04-01T00:14:46.891407+0000","last_undegraded":"2024-04-01T00:14:46.891407+0000","last_fullsized":"2024-04-01T00:14:46.891407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:40:31.530670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563905+0000","last_change":"2024-04-01T00:14:45.579154+0000","last_active":"2024-04-01T00:14:46.563905+0000","last_peered":"2024-04-01T00:14:46.563905+0000","last_clean":"2024-04-01T00:14:46.563905+0000","last_became_active":"2024-04-01T00:14:45.578267+0000","last_became_peered":"2024-04-01T00:14:45.578267+0000","last_unstale":"2024-04-01T00:14:46.563905+0000","last_undegraded":"2024-04-01T00:14:46.563905+0000","last_fullsized":"2024-04-01T00:14:46.563905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:18:00.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891084+0000","last_change":"2024-04-01T00:14:44.571485+0000","last_active":"2024-04-01T00:14:46.891084+0000","last_peered":"2024-04-01T00:14:46.891084+0000","last_clean":"2024-04-01T00:14:46.891084+0000","last_became_active":"2024-04-01T00:14:44.571204+0000","last_became_peered":"2024-04-01T00:14:44.571204+0000","last_unstale":"2024-04-01T00:14:46.891084+0000","last_undegraded":"2024-04-01T00:14:46.891084+0000","last_fullsized":"2024-04-01T00:14:46.891084+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:54:03.085286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614349+0000","last_change":"2024-04-01T00:14:45.584280+0000","last_active":"2024-04-01T00:14:46.614349+0000","last_peered":"2024-04-01T00:14:46.614349+0000","last_clean":"2024-04-01T00:14:46.614349+0000","last_became_active":"2024-04-01T00:14:45.583854+0000","last_became_peered":"2024-04-01T00:14:45.583854+0000","last_unstale":"2024-04-01T00:14:46.614349+0000","last_undegraded":"2024-04-01T00:14:46.614349+0000","last_fullsized":"2024-04-01T00:14:46.614349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:37:27.063558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890813+0000","last_change":"2024-04-01T00:14:44.581975+0000","last_active":"2024-04-01T00:14:46.890813+0000","last_peered":"2024-04-01T00:14:46.890813+0000","last_clean":"2024-04-01T00:14:46.890813+0000","last_became_active":"2024-04-01T00:14:44.581662+0000","last_became_peered":"2024-04-01T00:14:44.581662+0000","last_unstale":"2024-04-01T00:14:46.890813+0000","last_undegraded":"2024-04-01T00:14:46.890813+0000","last_fullsized":"2024-04-01T00:14:46.890813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:20.963361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567230+0000","last_change":"2024-04-01T00:14:45.573096+0000","last_active":"2024-04-01T00:14:46.567230+0000","last_peered":"2024-04-01T00:14:46.567230+0000","last_clean":"2024-04-01T00:14:46.567230+0000","last_became_active":"2024-04-01T00:14:45.572779+0000","last_became_peered":"2024-04-01T00:14:45.572779+0000","last_unstale":"2024-04-01T00:14:46.567230+0000","last_undegraded":"2024-04-01T00:14:46.567230+0000","last_fullsized":"2024-04-01T00:14:46.567230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:18:48.442342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890670+0000","last_change":"2024-04-01T00:14:44.574617+0000","last_active":"2024-04-01T00:14:46.890670+0000","last_peered":"2024-04-01T00:14:46.890670+0000","last_clean":"2024-04-01T00:14:46.890670+0000","last_became_active":"2024-04-01T00:14:44.574266+0000","last_became_peered":"2024-04-01T00:14:44.574266+0000","last_unstale":"2024-04-01T00:14:46.890670+0000","last_undegraded":"2024-04-01T00:14:46.890670+0000","last_fullsized":"2024-04-01T00:14:46.890670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:36:36.913544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890587+0000","last_change":"2024-04-01T00:14:45.579746+0000","last_active":"2024-04-01T00:14:46.890587+0000","last_peered":"2024-04-01T00:14:46.890587+0000","last_clean":"2024-04-01T00:14:46.890587+0000","last_became_active":"2024-04-01T00:14:45.579256+0000","last_became_peered":"2024-04-01T00:14:45.579256+0000","last_unstale":"2024-04-01T00:14:46.890587+0000","last_undegraded":"2024-04-01T00:14:46.890587+0000","last_fullsized":"2024-04-01T00:14:46.890587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:02:58.060301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891242+0000","last_change":"2024-04-01T00:14:44.577049+0000","last_active":"2024-04-01T00:14:46.891242+0000","last_peered":"2024-04-01T00:14:46.891242+0000","last_clean":"2024-04-01T00:14:46.891242+0000","last_became_active":"2024-04-01T00:14:44.576762+0000","last_became_peered":"2024-04-01T00:14:44.576762+0000","last_unstale":"2024-04-01T00:14:46.891242+0000","last_undegraded":"2024-04-01T00:14:46.891242+0000","last_fullsized":"2024-04-01T00:14:46.891242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:12:10.715316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563364+0000","last_change":"2024-04-01T00:14:45.576625+0000","last_active":"2024-04-01T00:14:46.563364+0000","last_peered":"2024-04-01T00:14:46.563364+0000","last_clean":"2024-04-01T00:14:46.563364+0000","last_became_active":"2024-04-01T00:14:45.576369+0000","last_became_peered":"2024-04-01T00:14:45.576369+0000","last_unstale":"2024-04-01T00:14:46.563364+0000","last_undegraded":"2024-04-01T00:14:46.563364+0000","last_fullsized":"2024-04-01T00:14:46.563364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:40:32.535953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890674+0000","last_change":"2024-04-01T00:14:44.578322+0000","last_active":"2024-04-01T00:14:46.890674+0000","last_peered":"2024-04-01T00:14:46.890674+0000","last_clean":"2024-04-01T00:14:46.890674+0000","last_became_active":"2024-04-01T00:14:44.578036+0000","last_became_peered":"2024-04-01T00:14:44.578036+0000","last_unstale":"2024-04-01T00:14:46.890674+0000","last_undegraded":"2024-04-01T00:14:46.890674+0000","last_fullsized":"2024-04-01T00:14:46.890674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:27:23.227338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567552+0000","last_change":"2024-04-01T00:14:45.571897+0000","last_active":"2024-04-01T00:14:46.567552+0000","last_peered":"2024-04-01T00:14:46.567552+0000","last_clean":"2024-04-01T00:14:46.567552+0000","last_became_active":"2024-04-01T00:14:45.571623+0000","last_became_peered":"2024-04-01T00:14:45.571623+0000","last_unstale":"2024-04-01T00:14:46.567552+0000","last_undegraded":"2024-04-01T00:14:46.567552+0000","last_fullsized":"2024-04-01T00:14:46.567552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:05:25.145272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891597+0000","last_change":"2024-04-01T00:14:44.581137+0000","last_active":"2024-04-01T00:14:46.891597+0000","last_peered":"2024-04-01T00:14:46.891597+0000","last_clean":"2024-04-01T00:14:46.891597+0000","last_became_active":"2024-04-01T00:14:44.580866+0000","last_became_peered":"2024-04-01T00:14:44.580866+0000","last_unstale":"2024-04-01T00:14:46.891597+0000","last_undegraded":"2024-04-01T00:14:46.891597+0000","last_fullsized":"2024-04-01T00:14:46.891597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:22:57.889531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891457+0000","last_change":"2024-04-01T00:14:45.574986+0000","last_active":"2024-04-01T00:14:46.891457+0000","last_peered":"2024-04-01T00:14:46.891457+0000","last_clean":"2024-04-01T00:14:46.891457+0000","last_became_active":"2024-04-01T00:14:45.574708+0000","last_became_peered":"2024-04-01T00:14:45.574708+0000","last_unstale":"2024-04-01T00:14:46.891457+0000","last_undegraded":"2024-04-01T00:14:46.891457+0000","last_fullsized":"2024-04-01T00:14:46.891457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:21:52.022463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566572+0000","last_change":"2024-04-01T00:14:44.581854+0000","last_active":"2024-04-01T00:14:46.566572+0000","last_peered":"2024-04-01T00:14:46.566572+0000","last_clean":"2024-04-01T00:14:46.566572+0000","last_became_active":"2024-04-01T00:14:44.581536+0000","last_became_peered":"2024-04-01T00:14:44.581536+0000","last_unstale":"2024-04-01T00:14:46.566572+0000","last_undegraded":"2024-04-01T00:14:46.566572+0000","last_fullsized":"2024-04-01T00:14:46.566572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:30:44.219648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891318+0000","last_change":"2024-04-01T00:14:45.586445+0000","last_active":"2024-04-01T00:14:46.891318+0000","last_peered":"2024-04-01T00:14:46.891318+0000","last_clean":"2024-04-01T00:14:46.891318+0000","last_became_active":"2024-04-01T00:14:45.586046+0000","last_became_peered":"2024-04-01T00:14:45.586046+0000","last_unstale":"2024-04-01T00:14:46.891318+0000","last_undegraded":"2024-04-01T00:14:46.891318+0000","last_fullsized":"2024-04-01T00:14:46.891318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:04:32.634685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567546+0000","last_change":"2024-04-01T00:14:44.578193+0000","last_active":"2024-04-01T00:14:46.567546+0000","last_peered":"2024-04-01T00:14:46.567546+0000","last_clean":"2024-04-01T00:14:46.567546+0000","last_became_active":"2024-04-01T00:14:44.577675+0000","last_became_peered":"2024-04-01T00:14:44.577675+0000","last_unstale":"2024-04-01T00:14:46.567546+0000","last_undegraded":"2024-04-01T00:14:46.567546+0000","last_fullsized":"2024-04-01T00:14:46.567546+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:03.887331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567438+0000","last_change":"2024-04-01T00:14:45.575566+0000","last_active":"2024-04-01T00:14:46.567438+0000","last_peered":"2024-04-01T00:14:46.567438+0000","last_clean":"2024-04-01T00:14:46.567438+0000","last_became_active":"2024-04-01T00:14:45.575308+0000","last_became_peered":"2024-04-01T00:14:45.575308+0000","last_unstale":"2024-04-01T00:14:46.567438+0000","last_undegraded":"2024-04-01T00:14:46.567438+0000","last_fullsized":"2024-04-01T00:14:46.567438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:39:53.898702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891296+0000","last_change":"2024-04-01T00:14:44.579376+0000","last_active":"2024-04-01T00:14:46.891296+0000","last_peered":"2024-04-01T00:14:46.891296+0000","last_clean":"2024-04-01T00:14:46.891296+0000","last_became_active":"2024-04-01T00:14:44.579048+0000","last_became_peered":"2024-04-01T00:14:44.579048+0000","last_unstale":"2024-04-01T00:14:46.891296+0000","last_undegraded":"2024-04-01T00:14:46.891296+0000","last_fullsized":"2024-04-01T00:14:46.891296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:05:54.873525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563432+0000","last_change":"2024-04-01T00:14:45.578771+0000","last_active":"2024-04-01T00:14:46.563432+0000","last_peered":"2024-04-01T00:14:46.563432+0000","last_clean":"2024-04-01T00:14:46.563432+0000","last_became_active":"2024-04-01T00:14:45.578489+0000","last_became_peered":"2024-04-01T00:14:45.578489+0000","last_unstale":"2024-04-01T00:14:46.563432+0000","last_undegraded":"2024-04-01T00:14:46.563432+0000","last_fullsized":"2024-04-01T00:14:46.563432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T02:22:58.894814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891038+0000","last_change":"2024-04-01T00:14:44.573445+0000","last_active":"2024-04-01T00:14:46.891038+0000","last_peered":"2024-04-01T00:14:46.891038+0000","last_clean":"2024-04-01T00:14:46.891038+0000","last_became_active":"2024-04-01T00:14:44.573175+0000","last_became_peered":"2024-04-01T00:14:44.573175+0000","last_unstale":"2024-04-01T00:14:46.891038+0000","last_undegraded":"2024-04-01T00:14:46.891038+0000","last_fullsized":"2024-04-01T00:14:46.891038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:16.713097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891544+0000","last_change":"2024-04-01T00:14:45.579113+0000","last_active":"2024-04-01T00:14:46.891544+0000","last_peered":"2024-04-01T00:14:46.891544+0000","last_clean":"2024-04-01T00:14:46.891544+0000","last_became_active":"2024-04-01T00:14:45.578810+0000","last_became_peered":"2024-04-01T00:14:45.578810+0000","last_unstale":"2024-04-01T00:14:46.891544+0000","last_undegraded":"2024-04-01T00:14:46.891544+0000","last_fullsized":"2024-04-01T00:14:46.891544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:58:35.474784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563381+0000","last_change":"2024-04-01T00:14:44.573020+0000","last_active":"2024-04-01T00:14:46.563381+0000","last_peered":"2024-04-01T00:14:46.563381+0000","last_clean":"2024-04-01T00:14:46.563381+0000","last_became_active":"2024-04-01T00:14:44.572756+0000","last_became_peered":"2024-04-01T00:14:44.572756+0000","last_unstale":"2024-04-01T00:14:46.563381+0000","last_undegraded":"2024-04-01T00:14:46.563381+0000","last_fullsized":"2024-04-01T00:14:46.563381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:30:44.219648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614408+0000","last_change":"2024-04-01T00:14:45.587064+0000","last_active":"2024-04-01T00:14:46.614408+0000","last_peered":"2024-04-01T00:14:46.614408+0000","last_clean":"2024-04-01T00:14:46.614408+0000","last_became_active":"2024-04-01T00:14:45.586811+0000","last_became_peered":"2024-04-01T00:14:45.586811+0000","last_unstale":"2024-04-01T00:14:46.614408+0000","last_undegraded":"2024-04-01T00:14:46.614408+0000","last_fullsized":"2024-04-01T00:14:46.614408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:45:46.245904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891139+0000","last_change":"2024-04-01T00:14:44.580598+0000","last_active":"2024-04-01T00:14:46.891139+0000","last_peered":"2024-04-01T00:14:46.891139+0000","last_clean":"2024-04-01T00:14:46.891139+0000","last_became_active":"2024-04-01T00:14:44.580245+0000","last_became_peered":"2024-04-01T00:14:44.580245+0000","last_unstale":"2024-04-01T00:14:46.891139+0000","last_undegraded":"2024-04-01T00:14:46.891139+0000","last_fullsized":"2024-04-01T00:14:46.891139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:01:11.993625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890731+0000","last_change":"2024-04-01T00:14:45.579184+0000","last_active":"2024-04-01T00:14:46.890731+0000","last_peered":"2024-04-01T00:14:46.890731+0000","last_clean":"2024-04-01T00:14:46.890731+0000","last_became_active":"2024-04-01T00:14:45.578927+0000","last_became_peered":"2024-04-01T00:14:45.578927+0000","last_unstale":"2024-04-01T00:14:46.890731+0000","last_undegraded":"2024-04-01T00:14:46.890731+0000","last_fullsized":"2024-04-01T00:14:46.890731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:25:04.892614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891033+0000","last_change":"2024-04-01T00:14:44.580512+0000","last_active":"2024-04-01T00:14:46.891033+0000","last_peered":"2024-04-01T00:14:46.891033+0000","last_clean":"2024-04-01T00:14:46.891033+0000","last_became_active":"2024-04-01T00:14:44.580132+0000","last_became_peered":"2024-04-01T00:14:44.580132+0000","last_unstale":"2024-04-01T00:14:46.891033+0000","last_undegraded":"2024-04-01T00:14:46.891033+0000","last_fullsized":"2024-04-01T00:14:46.891033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:45:06.475652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891231+0000","last_change":"2024-04-01T00:14:45.573040+0000","last_active":"2024-04-01T00:14:46.891231+0000","last_peered":"2024-04-01T00:14:46.891231+0000","last_clean":"2024-04-01T00:14:46.891231+0000","last_became_active":"2024-04-01T00:14:45.570675+0000","last_became_peered":"2024-04-01T00:14:45.570675+0000","last_unstale":"2024-04-01T00:14:46.891231+0000","last_undegraded":"2024-04-01T00:14:46.891231+0000","last_fullsized":"2024-04-01T00:14:46.891231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:01:12.998908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563879+0000","last_change":"2024-04-01T00:14:44.576879+0000","last_active":"2024-04-01T00:14:46.563879+0000","last_peered":"2024-04-01T00:14:46.563879+0000","last_clean":"2024-04-01T00:14:46.563879+0000","last_became_active":"2024-04-01T00:14:44.576567+0000","last_became_peered":"2024-04-01T00:14:44.576567+0000","last_unstale":"2024-04-01T00:14:46.563879+0000","last_undegraded":"2024-04-01T00:14:46.563879+0000","last_fullsized":"2024-04-01T00:14:46.563879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567675+0000","last_change":"2024-04-01T00:14:45.575820+0000","last_active":"2024-04-01T00:14:46.567675+0000","last_peered":"2024-04-01T00:14:46.567675+0000","last_clean":"2024-04-01T00:14:46.567675+0000","last_became_active":"2024-04-01T00:14:45.575545+0000","last_became_peered":"2024-04-01T00:14:45.575545+0000","last_unstale":"2024-04-01T00:14:46.567675+0000","last_undegraded":"2024-04-01T00:14:46.567675+0000","last_fullsized":"2024-04-01T00:14:46.567675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:37:27.063558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891231+0000","last_change":"2024-04-01T00:14:44.574281+0000","last_active":"2024-04-01T00:14:46.891231+0000","last_peered":"2024-04-01T00:14:46.891231+0000","last_clean":"2024-04-01T00:14:46.891231+0000","last_became_active":"2024-04-01T00:14:44.573981+0000","last_became_peered":"2024-04-01T00:14:44.573981+0000","last_unstale":"2024-04-01T00:14:46.891231+0000","last_undegraded":"2024-04-01T00:14:46.891231+0000","last_fullsized":"2024-04-01T00:14:46.891231+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:15:02.632874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563654+0000","last_change":"2024-04-01T00:14:45.578706+0000","last_active":"2024-04-01T00:14:46.563654+0000","last_peered":"2024-04-01T00:14:46.563654+0000","last_clean":"2024-04-01T00:14:46.563654+0000","last_became_active":"2024-04-01T00:14:45.578438+0000","last_became_peered":"2024-04-01T00:14:45.578438+0000","last_unstale":"2024-04-01T00:14:46.563654+0000","last_undegraded":"2024-04-01T00:14:46.563654+0000","last_fullsized":"2024-04-01T00:14:46.563654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:25:08.051094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891298+0000","last_change":"2024-04-01T00:14:44.574997+0000","last_active":"2024-04-01T00:14:46.891298+0000","last_peered":"2024-04-01T00:14:46.891298+0000","last_clean":"2024-04-01T00:14:46.891298+0000","last_became_active":"2024-04-01T00:14:44.574734+0000","last_became_peered":"2024-04-01T00:14:44.574734+0000","last_unstale":"2024-04-01T00:14:46.891298+0000","last_undegraded":"2024-04-01T00:14:46.891298+0000","last_fullsized":"2024-04-01T00:14:46.891298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:12:12.698377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890676+0000","last_change":"2024-04-01T00:14:45.587486+0000","last_active":"2024-04-01T00:14:46.890676+0000","last_peered":"2024-04-01T00:14:46.890676+0000","last_clean":"2024-04-01T00:14:46.890676+0000","last_became_active":"2024-04-01T00:14:45.587189+0000","last_became_peered":"2024-04-01T00:14:45.587189+0000","last_unstale":"2024-04-01T00:14:46.890676+0000","last_undegraded":"2024-04-01T00:14:46.890676+0000","last_fullsized":"2024-04-01T00:14:46.890676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:27.615615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891183+0000","last_change":"2024-04-01T00:14:44.576591+0000","last_active":"2024-04-01T00:14:46.891183+0000","last_peered":"2024-04-01T00:14:46.891183+0000","last_clean":"2024-04-01T00:14:46.891183+0000","last_became_active":"2024-04-01T00:14:44.576336+0000","last_became_peered":"2024-04-01T00:14:44.576336+0000","last_unstale":"2024-04-01T00:14:46.891183+0000","last_undegraded":"2024-04-01T00:14:46.891183+0000","last_fullsized":"2024-04-01T00:14:46.891183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:00:27.174967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890562+0000","last_change":"2024-04-01T00:14:45.574998+0000","last_active":"2024-04-01T00:14:46.890562+0000","last_peered":"2024-04-01T00:14:46.890562+0000","last_clean":"2024-04-01T00:14:46.890562+0000","last_became_active":"2024-04-01T00:14:45.574709+0000","last_became_peered":"2024-04-01T00:14:45.574709+0000","last_unstale":"2024-04-01T00:14:46.890562+0000","last_undegraded":"2024-04-01T00:14:46.890562+0000","last_fullsized":"2024-04-01T00:14:46.890562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T06:42:21.968644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566495+0000","last_change":"2024-04-01T00:14:44.579815+0000","last_active":"2024-04-01T00:14:46.566495+0000","last_peered":"2024-04-01T00:14:46.566495+0000","last_clean":"2024-04-01T00:14:46.566495+0000","last_became_active":"2024-04-01T00:14:44.579502+0000","last_became_peered":"2024-04-01T00:14:44.579502+0000","last_unstale":"2024-04-01T00:14:46.566495+0000","last_undegraded":"2024-04-01T00:14:46.566495+0000","last_fullsized":"2024-04-01T00:14:46.566495+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890721+0000","last_change":"2024-04-01T00:14:45.576263+0000","last_active":"2024-04-01T00:14:46.890721+0000","last_peered":"2024-04-01T00:14:46.890721+0000","last_clean":"2024-04-01T00:14:46.890721+0000","last_became_active":"2024-04-01T00:14:45.575986+0000","last_became_peered":"2024-04-01T00:14:45.575986+0000","last_unstale":"2024-04-01T00:14:46.890721+0000","last_undegraded":"2024-04-01T00:14:46.890721+0000","last_fullsized":"2024-04-01T00:14:46.890721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:08.841082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890823+0000","last_change":"2024-04-01T00:14:44.579144+0000","last_active":"2024-04-01T00:14:46.890823+0000","last_peered":"2024-04-01T00:14:46.890823+0000","last_clean":"2024-04-01T00:14:46.890823+0000","last_became_active":"2024-04-01T00:14:44.578861+0000","last_became_peered":"2024-04-01T00:14:44.578861+0000","last_unstale":"2024-04-01T00:14:46.890823+0000","last_undegraded":"2024-04-01T00:14:46.890823+0000","last_fullsized":"2024-04-01T00:14:46.890823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:43:07.835799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890469+0000","last_change":"2024-04-01T00:14:45.572345+0000","last_active":"2024-04-01T00:14:46.890469+0000","last_peered":"2024-04-01T00:14:46.890469+0000","last_clean":"2024-04-01T00:14:46.890469+0000","last_became_active":"2024-04-01T00:14:45.572079+0000","last_became_peered":"2024-04-01T00:14:45.572079+0000","last_unstale":"2024-04-01T00:14:46.890469+0000","last_undegraded":"2024-04-01T00:14:46.890469+0000","last_fullsized":"2024-04-01T00:14:46.890469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T08:36:37.918827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890952+0000","last_change":"2024-04-01T00:14:44.579438+0000","last_active":"2024-04-01T00:14:46.890952+0000","last_peered":"2024-04-01T00:14:46.890952+0000","last_clean":"2024-04-01T00:14:46.890952+0000","last_became_active":"2024-04-01T00:14:44.579181+0000","last_became_peered":"2024-04-01T00:14:44.579181+0000","last_unstale":"2024-04-01T00:14:46.890952+0000","last_undegraded":"2024-04-01T00:14:46.890952+0000","last_fullsized":"2024-04-01T00:14:46.890952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T03:09:38.633625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614308+0000","last_change":"2024-04-01T00:14:45.584589+0000","last_active":"2024-04-01T00:14:46.614308+0000","last_peered":"2024-04-01T00:14:46.614308+0000","last_clean":"2024-04-01T00:14:46.614308+0000","last_became_active":"2024-04-01T00:14:45.583808+0000","last_became_peered":"2024-04-01T00:14:45.583808+0000","last_unstale":"2024-04-01T00:14:46.614308+0000","last_undegraded":"2024-04-01T00:14:46.614308+0000","last_fullsized":"2024-04-01T00:14:46.614308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:05:25.145272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566981+0000","last_change":"2024-04-01T00:14:44.577757+0000","last_active":"2024-04-01T00:14:46.566981+0000","last_peered":"2024-04-01T00:14:46.566981+0000","last_clean":"2024-04-01T00:14:46.566981+0000","last_became_active":"2024-04-01T00:14:44.577431+0000","last_became_peered":"2024-04-01T00:14:44.577431+0000","last_unstale":"2024-04-01T00:14:46.566981+0000","last_undegraded":"2024-04-01T00:14:46.566981+0000","last_fullsized":"2024-04-01T00:14:46.566981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:19:31.290107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890567+0000","last_change":"2024-04-01T00:14:45.575462+0000","last_active":"2024-04-01T00:14:46.890567+0000","last_peered":"2024-04-01T00:14:46.890567+0000","last_clean":"2024-04-01T00:14:46.890567+0000","last_became_active":"2024-04-01T00:14:45.575204+0000","last_became_peered":"2024-04-01T00:14:45.575204+0000","last_unstale":"2024-04-01T00:14:46.890567+0000","last_undegraded":"2024-04-01T00:14:46.890567+0000","last_fullsized":"2024-04-01T00:14:46.890567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:12:17.718380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563922+0000","last_change":"2024-04-01T00:14:44.567605+0000","last_active":"2024-04-01T00:14:46.563922+0000","last_peered":"2024-04-01T00:14:46.563922+0000","last_clean":"2024-04-01T00:14:46.563922+0000","last_became_active":"2024-04-01T00:14:44.567238+0000","last_became_peered":"2024-04-01T00:14:44.567238+0000","last_unstale":"2024-04-01T00:14:46.563922+0000","last_undegraded":"2024-04-01T00:14:46.563922+0000","last_fullsized":"2024-04-01T00:14:46.563922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:29:42.707898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.567782+0000","last_change":"2024-04-01T00:14:45.574780+0000","last_active":"2024-04-01T00:14:46.567782+0000","last_peered":"2024-04-01T00:14:46.567782+0000","last_clean":"2024-04-01T00:14:46.567782+0000","last_became_active":"2024-04-01T00:14:45.574502+0000","last_became_peered":"2024-04-01T00:14:45.574502+0000","last_unstale":"2024-04-01T00:14:46.567782+0000","last_undegraded":"2024-04-01T00:14:46.567782+0000","last_fullsized":"2024-04-01T00:14:46.567782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:59:27.154412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566607+0000","last_change":"2024-04-01T00:14:44.579442+0000","last_active":"2024-04-01T00:14:46.566607+0000","last_peered":"2024-04-01T00:14:46.566607+0000","last_clean":"2024-04-01T00:14:46.566607+0000","last_became_active":"2024-04-01T00:14:44.579151+0000","last_became_peered":"2024-04-01T00:14:44.579151+0000","last_unstale":"2024-04-01T00:14:46.566607+0000","last_undegraded":"2024-04-01T00:14:46.566607+0000","last_fullsized":"2024-04-01T00:14:46.566607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:16:07.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566381+0000","last_change":"2024-04-01T00:14:45.576368+0000","last_active":"2024-04-01T00:14:46.566381+0000","last_peered":"2024-04-01T00:14:46.566381+0000","last_clean":"2024-04-01T00:14:46.566381+0000","last_became_active":"2024-04-01T00:14:45.576068+0000","last_became_peered":"2024-04-01T00:14:45.576068+0000","last_unstale":"2024-04-01T00:14:46.566381+0000","last_undegraded":"2024-04-01T00:14:46.566381+0000","last_fullsized":"2024-04-01T00:14:46.566381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:02:58.060301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563713+0000","last_change":"2024-04-01T00:14:44.570761+0000","last_active":"2024-04-01T00:14:46.563713+0000","last_peered":"2024-04-01T00:14:46.563713+0000","last_clean":"2024-04-01T00:14:46.563713+0000","last_became_active":"2024-04-01T00:14:44.569379+0000","last_became_peered":"2024-04-01T00:14:44.569379+0000","last_unstale":"2024-04-01T00:14:46.563713+0000","last_undegraded":"2024-04-01T00:14:46.563713+0000","last_fullsized":"2024-04-01T00:14:46.563713+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T10:20:00.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890833+0000","last_change":"2024-04-01T00:14:45.578455+0000","last_active":"2024-04-01T00:14:46.890833+0000","last_peered":"2024-04-01T00:14:46.890833+0000","last_clean":"2024-04-01T00:14:46.890833+0000","last_became_active":"2024-04-01T00:14:45.578139+0000","last_became_peered":"2024-04-01T00:14:45.578139+0000","last_unstale":"2024-04-01T00:14:46.890833+0000","last_undegraded":"2024-04-01T00:14:46.890833+0000","last_fullsized":"2024-04-01T00:14:46.890833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T01:02:58.060301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566588+0000","last_change":"2024-04-01T00:14:44.579166+0000","last_active":"2024-04-01T00:14:46.566588+0000","last_peered":"2024-04-01T00:14:46.566588+0000","last_clean":"2024-04-01T00:14:46.566588+0000","last_became_active":"2024-04-01T00:14:44.578892+0000","last_became_peered":"2024-04-01T00:14:44.578892+0000","last_unstale":"2024-04-01T00:14:46.566588+0000","last_undegraded":"2024-04-01T00:14:46.566588+0000","last_fullsized":"2024-04-01T00:14:46.566588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:08:46.388379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.566443+0000","last_change":"2024-04-01T00:14:45.579387+0000","last_active":"2024-04-01T00:14:46.566443+0000","last_peered":"2024-04-01T00:14:46.566443+0000","last_clean":"2024-04-01T00:14:46.566443+0000","last_became_active":"2024-04-01T00:14:45.579105+0000","last_became_peered":"2024-04-01T00:14:45.579105+0000","last_unstale":"2024-04-01T00:14:46.566443+0000","last_undegraded":"2024-04-01T00:14:46.566443+0000","last_fullsized":"2024-04-01T00:14:46.566443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:45:07.480935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.563481+0000","last_change":"2024-04-01T00:14:44.568541+0000","last_active":"2024-04-01T00:14:46.563481+0000","last_peered":"2024-04-01T00:14:46.563481+0000","last_clean":"2024-04-01T00:14:46.563481+0000","last_became_active":"2024-04-01T00:14:44.568266+0000","last_became_peered":"2024-04-01T00:14:44.568266+0000","last_unstale":"2024-04-01T00:14:46.563481+0000","last_undegraded":"2024-04-01T00:14:46.563481+0000","last_fullsized":"2024-04-01T00:14:46.563481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T00:51:48.916899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890934+0000","last_change":"2024-04-01T00:14:45.575599+0000","last_active":"2024-04-01T00:14:46.890934+0000","last_peered":"2024-04-01T00:14:46.890934+0000","last_clean":"2024-04-01T00:14:46.890934+0000","last_became_active":"2024-04-01T00:14:45.575325+0000","last_became_peered":"2024-04-01T00:14:45.575325+0000","last_unstale":"2024-04-01T00:14:46.890934+0000","last_undegraded":"2024-04-01T00:14:46.890934+0000","last_fullsized":"2024-04-01T00:14:46.890934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T11:05:51.351096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890980+0000","last_change":"2024-04-01T00:14:44.571915+0000","last_active":"2024-04-01T00:14:46.890980+0000","last_peered":"2024-04-01T00:14:46.890980+0000","last_clean":"2024-04-01T00:14:46.890980+0000","last_became_active":"2024-04-01T00:14:44.571316+0000","last_became_peered":"2024-04-01T00:14:44.571316+0000","last_unstale":"2024-04-01T00:14:46.890980+0000","last_undegraded":"2024-04-01T00:14:46.890980+0000","last_fullsized":"2024-04-01T00:14:46.890980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:12:26.610333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890927+0000","last_change":"2024-04-01T00:14:45.577678+0000","last_active":"2024-04-01T00:14:46.890927+0000","last_peered":"2024-04-01T00:14:46.890927+0000","last_clean":"2024-04-01T00:14:46.890927+0000","last_became_active":"2024-04-01T00:14:45.577408+0000","last_became_peered":"2024-04-01T00:14:45.577408+0000","last_unstale":"2024-04-01T00:14:46.890927+0000","last_undegraded":"2024-04-01T00:14:46.890927+0000","last_fullsized":"2024-04-01T00:14:46.890927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T05:45:07.480935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.891136+0000","last_change":"2024-04-01T00:14:44.578180+0000","last_active":"2024-04-01T00:14:46.891136+0000","last_peered":"2024-04-01T00:14:46.891136+0000","last_clean":"2024-04-01T00:14:46.891136+0000","last_became_active":"2024-04-01T00:14:44.577563+0000","last_became_peered":"2024-04-01T00:14:44.577563+0000","last_unstale":"2024-04-01T00:14:46.891136+0000","last_undegraded":"2024-04-01T00:14:46.891136+0000","last_fullsized":"2024-04-01T00:14:46.891136+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T12:09:59.040691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614483+0000","last_change":"2024-04-01T00:14:45.584398+0000","last_active":"2024-04-01T00:14:46.614483+0000","last_peered":"2024-04-01T00:14:46.614483+0000","last_clean":"2024-04-01T00:14:46.614483+0000","last_became_active":"2024-04-01T00:14:45.584084+0000","last_became_peered":"2024-04-01T00:14:45.584084+0000","last_unstale":"2024-04-01T00:14:46.614483+0000","last_undegraded":"2024-04-01T00:14:46.614483+0000","last_fullsized":"2024-04-01T00:14:46.614483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:44.553544+0000","last_clean_scrub_stamp":"2024-04-01T00:14:44.553544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T04:03:08.207324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.890567+0000","last_change":"2024-04-01T00:14:44.581624+0000","last_active":"2024-04-01T00:14:46.890567+0000","last_peered":"2024-04-01T00:14:46.890567+0000","last_clean":"2024-04-01T00:14:46.890567+0000","last_became_active":"2024-04-01T00:14:44.581339+0000","last_became_peered":"2024-04-01T00:14:44.581339+0000","last_unstale":"2024-04-01T00:14:46.890567+0000","last_undegraded":"2024-04-01T00:14:46.890567+0000","last_fullsized":"2024-04-01T00:14:46.890567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:14:43.548261+0000","last_clean_scrub_stamp":"2024-04-01T00:14:43.548261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T09:01:02.639733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-01T00:14:46.614639+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T00:14:46.614639+0000","last_peered":"2024-04-01T00:14:46.614639+0000","last_clean":"2024-04-01T00:14:46.614639+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T00:14:46.614639+0000","last_undegraded":"2024-04-01T00:14:46.614639+0000","last_fullsized":"2024-04-01T00:14:46.614639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T00:14:53.341 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-01T00:14:53.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-01T00:14:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.483+0000 7f358fec2640 1 Processor -- start 2024-04-01T00:14:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.483+0000 7f358fec2640 1 -- start start 2024-04-01T00:14:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358fec2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 0x7f35880cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358fec2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f35880b1140 0x7f35880b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358fec2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35880cc730 con 0x7f35880cf740 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358fec2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f35880b1ae0 con 0x7f35880b1140 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358fec2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f35880b0f80 con 0x7f35880b0ba0 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358e45f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f35880cf740 0x7f35880cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48878/0 (socket says 172.21.15.64:48878) 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358e45f640 1 -- 172.21.15.64:0/1769425058 learned_addr learned my addr 172.21.15.64:0/1769425058 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358d45d640 1 --2- 172.21.15.64:0/1769425058 >> v2:172.21.15.100:3300/0 conn(0x7f35880b1140 0x7f35880b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358dc5e640 1 --2- 172.21.15.64:0/1769425058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 0x7f35880cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358cc5c640 1 -- 172.21.15.64:0/1769425058 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3154127715 0 0) 0x7f35880cc730 con 0x7f35880cf740 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358cc5c640 1 -- 172.21.15.64:0/1769425058 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3574003520 con 0x7f35880cf740 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358dc5e640 1 -- 172.21.15.64:0/1769425058 >> v2:172.21.15.100:3300/0 conn(0x7f35880b1140 msgr2=0x7f35880b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358dc5e640 1 --2- 172.21.15.64:0/1769425058 >> v2:172.21.15.100:3300/0 conn(0x7f35880b1140 0x7f35880b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358dc5e640 1 -- 172.21.15.64:0/1769425058 >> v1:172.21.15.64:6789/0 conn(0x7f35880cf740 legacy=0x7f35880cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.484+0000 7f358dc5e640 1 -- 172.21.15.64:0/1769425058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35880cca10 con 0x7f35880b0ba0 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.485+0000 7f358dc5e640 1 --2- 172.21.15.64:0/1769425058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 0x7f35880cd7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35840097a0 tx=0x7f358402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614c25a09745adf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.485+0000 7f358cc5c640 1 -- 172.21.15.64:0/1769425058 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584035020 con 0x7f35880b0ba0 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.485+0000 7f358cc5c640 1 -- 172.21.15.64:0/1769425058 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3584031830 con 0x7f35880b0ba0 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.485+0000 7f358cc5c640 1 -- 172.21.15.64:0/1769425058 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584031be0 con 0x7f35880b0ba0 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 -- 172.21.15.64:0/1769425058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 msgr2=0x7f35880cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 --2- 172.21.15.64:0/1769425058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 0x7f35880cd7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35840097a0 tx=0x7f358402b8d0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 -- 172.21.15.64:0/1769425058 shutdown_connections 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 --2- 172.21.15.64:0/1769425058 >> v2:172.21.15.100:3300/0 conn(0x7f35880b1140 0x7f35880b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 --2- 172.21.15.64:0/1769425058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b0ba0 0x7f35880cd7d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 -- 172.21.15.64:0/1769425058 >> 172.21.15.64:0/1769425058 conn(0x7f3588072810 msgr2=0x7f3588072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 -- 172.21.15.64:0/1769425058 shutdown_connections 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.486+0000 7f358fec2640 1 -- 172.21.15.64:0/1769425058 wait complete. 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.487+0000 7f358fec2640 1 Processor -- start 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.487+0000 7f358fec2640 1 -- start start 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35880b0ba0 0x7f35880c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b1140 0x7f35880c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35880cd2b0 con 0x7f35880cf740 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35880ccfd0 con 0x7f35880b0ba0 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358fec2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3588162e10 con 0x7f35880b1140 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38646/0 (socket says 172.21.15.64:38646) 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 -- 172.21.15.64:0/3733178739 learned_addr learned my addr 172.21.15.64:0/3733178739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358d45d640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b1140 0x7f35880c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358dc5e640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35880b0ba0 0x7f35880c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 -- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b1140 msgr2=0x7f35880c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b1140 0x7f35880c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 -- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35880b0ba0 msgr2=0x7f35880c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35880b0ba0 0x7f35880c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 -- 172.21.15.64:0/3733178739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35880c54a0 con 0x7f35880cf740 2024-04-01T00:14:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.488+0000 7f358e45f640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f358000cb40 tx=0x7f358000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.489+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580017070 con 0x7f35880cf740 2024-04-01T00:14:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.489+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358000aca0 con 0x7f35880cf740 2024-04-01T00:14:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.489+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35880c5760 con 0x7f35880cf740 2024-04-01T00:14:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.489+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580012620 con 0x7f35880cf740 2024-04-01T00:14:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.489+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35880c5d20 con 0x7f35880cf740 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.490+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35880681a0 con 0x7f35880cf740 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.490+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f358001a070 con 0x7f35880cf740 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.490+0000 7f357e7fc640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 0x7f356c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.490+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f358008a580 con 0x7f35880cf740 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.491+0000 7f358dc5e640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 0x7f356c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.491+0000 7f358dc5e640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 0x7f356c06de50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3584036000 tx=0x7f358403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.495+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3580058b40 con 0x7f35880cf740 2024-04-01T00:14:53.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.749+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f35880c6fa0 con 0x7f35880cf740 2024-04-01T00:14:53.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.750+0000 7f357e7fc640 1 -- 172.21.15.64:0/3733178739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f358005cbe0 con 0x7f35880cf740 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAB/AlmQSrFDBAAIkPmgw3eT122JxbfaL7NsQ== 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-01T00:14:53.753 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2024-04-01T00:14:53.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 msgr2=0x7f356c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 0x7f356c06de50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3584036000 tx=0x7f358403c040 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 msgr2=0x7f35880c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f358000cb40 tx=0x7f358000a060 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 shutdown_connections 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f356c06b990 0x7f356c06de50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35880cf740 0x7f35880c74b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35880b1140 0x7f35880c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 --2- 172.21.15.64:0/3733178739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35880b0ba0 0x7f35880c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.752+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 >> 172.21.15.64:0/3733178739 conn(0x7f3588072810 msgr2=0x7f358806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.753+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 shutdown_connections 2024-04-01T00:14:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.753+0000 7f358fec2640 1 -- 172.21.15.64:0/3733178739 wait complete. 2024-04-01T00:14:53.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T00:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.865+0000 7fb992647640 1 Processor -- start 2024-04-01T00:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.865+0000 7fb992647640 1 -- start start 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb992647640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb98c15e5b0 0x7fb98c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb992647640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98c15bf50 con 0x7fb98c0c9cf0 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb992647640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98c15eb90 con 0x7fb98c15bb70 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb992647640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c0afe60 con 0x7fb98c15e5b0 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb98c15e5b0 0x7fb98c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb98c15e5b0 0x7fb98c06d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38660/0 (socket says 172.21.15.64:38660) 2024-04-01T00:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb990be4640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb98c0c9cf0 0x7fb98c06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48892/0 (socket says 172.21.15.64:48892) 2024-04-01T00:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 -- 172.21.15.64:0/1985469420 learned_addr learned my addr 172.21.15.64:0/1985469420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 -- 172.21.15.64:0/1985469420 >> v1:172.21.15.100:6789/0 conn(0x7fb98c15bb70 legacy=0x7fb98c071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 -- 172.21.15.64:0/1985469420 >> v1:172.21.15.64:6789/0 conn(0x7fb98c0c9cf0 legacy=0x7fb98c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.866+0000 7fb98b7fe640 1 -- 172.21.15.64:0/1985469420 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c15ee70 con 0x7fb98c15e5b0 2024-04-01T00:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.867+0000 7fb98b7fe640 1 --2- 172.21.15.64:0/1985469420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15e5b0 0x7fb98c06d560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3db0d43154a82e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.867+0000 7fb98affd640 1 -- 172.21.15.64:0/1985469420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c02f840 con 0x7fb98c15e5b0 2024-04-01T00:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.867+0000 7fb98affd640 1 -- 172.21.15.64:0/1985469420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c02f9e0 con 0x7fb98c15e5b0 2024-04-01T00:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.867+0000 7fb98affd640 1 -- 172.21.15.64:0/1985469420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c02fdb0 con 0x7fb98c15e5b0 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 -- 172.21.15.64:0/1985469420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15e5b0 msgr2=0x7fb98c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 --2- 172.21.15.64:0/1985469420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15e5b0 0x7fb98c06d560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c02ba20 comp rx=0 tx=0).stop 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 -- 172.21.15.64:0/1985469420 shutdown_connections 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 --2- 172.21.15.64:0/1985469420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15e5b0 0x7fb98c06d560 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 -- 172.21.15.64:0/1985469420 >> 172.21.15.64:0/1985469420 conn(0x7fb98c0d03e0 msgr2=0x7fb98c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 -- 172.21.15.64:0/1985469420 shutdown_connections 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.868+0000 7fb992647640 1 -- 172.21.15.64:0/1985469420 wait complete. 2024-04-01T00:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.869+0000 7fb992647640 1 Processor -- start 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.869+0000 7fb992647640 1 -- start start 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98c0c9cf0 0x7fb98c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb98c15e5b0 0x7fb98c147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c071c50 con 0x7fb98c15bb70 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0c97d0 con 0x7fb98c0c9cf0 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c15f430 con 0x7fb98c15e5b0 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38674/0 (socket says 172.21.15.64:38674) 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 -- 172.21.15.64:0/3812662085 learned_addr learned my addr 172.21.15.64:0/3812662085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98bfff640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98c0c9cf0 0x7fb98c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb990be4640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb98c15e5b0 0x7fb98c147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 -- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb98c15e5b0 msgr2=0x7fb98c147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb98c15e5b0 0x7fb98c147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 -- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98c0c9cf0 msgr2=0x7fb98c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98c0c9cf0 0x7fb98c0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 -- 172.21.15.64:0/3812662085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1f8360 con 0x7fb98c15bb70 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb98b7fe640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb97c02ba60 tx=0x7fb97c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c047070 con 0x7fb98c15bb70 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c0423f0 con 0x7fb98c15bb70 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.870+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c1f77e0 con 0x7fb98c15bb70 2024-04-01T00:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.871+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0427e0 con 0x7fb98c15bb70 2024-04-01T00:14:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.871+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1f8080 con 0x7fb98c15bb70 2024-04-01T00:14:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.871+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c1f7bd0 con 0x7fb98c15bb70 2024-04-01T00:14:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.872+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb97c04a070 con 0x7fb98c15bb70 2024-04-01T00:14:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.872+0000 7fb988ff9640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 0x7fb96c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.872+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb97c0ac840 con 0x7fb98c15bb70 2024-04-01T00:14:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.873+0000 7fb98bfff640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 0x7fb96c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.873+0000 7fb98bfff640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 0x7fb96c06de50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb974000f60 tx=0x7fb974009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:53.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:53.877+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb97c033030 con 0x7fb98c15bb70 2024-04-01T00:14:54.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.096+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98c15fb70 con 0x7fb98c15bb70 2024-04-01T00:14:54.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.096+0000 7fb988ff9640 1 -- 172.21.15.64:0/3812662085 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7fb97c07ae90 con 0x7fb98c15bb70 2024-04-01T00:14:54.098 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:54.098 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T00:14:54.099 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 msgr2=0x7fb96c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 0x7fb96c06de50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb974000f60 tx=0x7fb974009040 comp rx=0 tx=0).stop 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 msgr2=0x7fb98c155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb97c02ba60 tx=0x7fb97c004730 comp rx=0 tx=0).stop 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 shutdown_connections 2024-04-01T00:14:54.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb96c06b990 0x7fb96c06de50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb98c15e5b0 0x7fb98c147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb98c15bb70 0x7fb98c155b70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 --2- 172.21.15.64:0/3812662085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98c0c9cf0 0x7fb98c0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 >> 172.21.15.64:0/3812662085 conn(0x7fb98c0d03e0 msgr2=0x7fb98c155190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 shutdown_connections 2024-04-01T00:14:54.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.098+0000 7fb992647640 1 -- 172.21.15.64:0/3812662085 wait complete. 2024-04-01T00:14:54.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-01T00:14:44.927110+0000', 'modified': '2024-04-01T00:14:46.968725+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5224': {'gid': 5224, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/1975239378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 1975239378}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1975239378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5224, 'qdb_cluster': [5224]} 2024-04-01T00:14:54.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-01T00:14:54.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-01T00:14:54.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 Processor -- start 2024-04-01T00:14:54.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 -- start start 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41d80c5c50 0x7f41d80cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d806dbf0 con 0x7f41d80c6500 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80cb3b0 con 0x7f41d80cd660 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.617+0000 7f41df1ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80cb690 con 0x7f41d80c5c50 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41d7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f41d80c6500 0x7f41d80c5540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48900/0 (socket says 172.21.15.64:48900) 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41d7fff640 1 -- 172.21.15.64:0/4157253033 learned_addr learned my addr 172.21.15.64:0/4157253033 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41dd749640 1 --2- 172.21.15.64:0/4157253033 >> v2:172.21.15.100:3300/0 conn(0x7f41d80c5c50 0x7f41d80cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375532208 0 0) 0x7f41d806dbf0 con 0x7f41d80c6500 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c8003520 con 0x7f41d80c6500 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41dd749640 1 -- 172.21.15.64:0/4157253033 >> v1:172.21.15.100:6789/0 conn(0x7f41d80cd660 legacy=0x7f41d806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41dd749640 1 -- 172.21.15.64:0/4157253033 >> v1:172.21.15.64:6789/0 conn(0x7f41d80c6500 legacy=0x7f41d80c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573348752 0 0) 0x7f41d80cb3b0 con 0x7f41d80cd660 2024-04-01T00:14:54.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.618+0000 7f41dd749640 1 -- 172.21.15.64:0/4157253033 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80c4a60 con 0x7f41d80c5c50 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41dd749640 1 --2- 172.21.15.64:0/4157253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80c5c50 0x7f41d80cadd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009d90 tx=0x7f41d002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bceadd8fcb478a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0039070 con 0x7f41d80c5c50 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d002cbd0 con 0x7f41d80c5c50 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41d77fe640 1 -- 172.21.15.64:0/4157253033 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0034440 con 0x7f41d80c5c50 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41df1ac640 1 -- 172.21.15.64:0/4157253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80c5c50 msgr2=0x7f41d80cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.619+0000 7f41df1ac640 1 --2- 172.21.15.64:0/4157253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80c5c50 0x7f41d80cadd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009d90 tx=0x7f41d002be20 comp rx=0 tx=0).stop 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.620+0000 7f41df1ac640 1 -- 172.21.15.64:0/4157253033 shutdown_connections 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.620+0000 7f41df1ac640 1 --2- 172.21.15.64:0/4157253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80c5c50 0x7f41d80cadd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.620+0000 7f41df1ac640 1 -- 172.21.15.64:0/4157253033 >> 172.21.15.64:0/4157253033 conn(0x7f41d80cfa90 msgr2=0x7f41d80cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.620+0000 7f41df1ac640 1 -- 172.21.15.64:0/4157253033 shutdown_connections 2024-04-01T00:14:54.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.620+0000 7f41df1ac640 1 -- 172.21.15.64:0/4157253033 wait complete. 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.621+0000 7f41df1ac640 1 Processor -- start 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.621+0000 7f41df1ac640 1 -- start start 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.621+0000 7f41df1ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d80c5c50 0x7f41d8072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 0x7f41d8149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80cc0a0 con 0x7f41d80c6500 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80c50b0 con 0x7f41d80cd660 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f41d806cc50 con 0x7f41d80c5c50 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38678/0 (socket says 172.21.15.64:38678) 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 -- 172.21.15.64:0/2484653283 learned_addr learned my addr 172.21.15.64:0/2484653283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41dd749640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 0x7f41d8149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41dcf48640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d80c5c50 0x7f41d8072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 -- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d80c5c50 msgr2=0x7f41d8072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d80c5c50 0x7f41d8072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 -- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 msgr2=0x7f41d8149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 0x7f41d8149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 -- 172.21.15.64:0/2484653283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d8072a60 con 0x7f41d80c6500 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41dd749640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 0x7f41d8149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c530 tx=0x7f41c400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d57fa640 1 -- 172.21.15.64:0/2484653283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4017070 con 0x7f41d80c6500 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d57fa640 1 -- 172.21.15.64:0/2484653283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c4005ce0 con 0x7f41d80c6500 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d8143370 con 0x7f41d80c6500 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41d57fa640 1 -- 172.21.15.64:0/2484653283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c40072b0 con 0x7f41d80c6500 2024-04-01T00:14:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.622+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81444c0 con 0x7f41d80c6500 2024-04-01T00:14:54.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.624+0000 7f41d57fa640 1 -- 172.21.15.64:0/2484653283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f41c401a070 con 0x7f41d80c6500 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.624+0000 7f41d57fa640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 0x7f41bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.624+0000 7f41d57fa640 1 -- 172.21.15.64:0/2484653283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f41c4089b70 con 0x7f41d80c6500 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.624+0000 7f41dcf48640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 0x7f41bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.625+0000 7f41dcf48640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 0x7f41bc06de50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000f60 tx=0x7f41cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.625+0000 7f41e1d8f740 1 Processor -- start 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.625+0000 7f41e1d8f740 1 -- start start 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6fae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6f604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5642e6ec9af0 con 0x5642e6f8db20 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5642e6e729d0 con 0x5642e6fadef0 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5642e6ec5b90 con 0x5642e6fad610 2024-04-01T00:14:54.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38686/0 (socket says 172.21.15.64:38686) 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 -- 172.21.15.64:0/3379172970 learned_addr learned my addr 172.21.15.64:0/3379172970 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41d7fff640 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6fae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dd749640 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6f604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 -- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 msgr2=0x5642e6f604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6f604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 -- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 msgr2=0x5642e6fae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6fae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 -- 172.21.15.64:0/3379172970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e6bde2d0 con 0x5642e6f8db20 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41dcf48640 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00fc00 tx=0x7f41cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7e023fe39632fc server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41c27fc640 1 -- 172.21.15.64:0/3379172970 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc025070 con 0x5642e6f8db20 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41c27fc640 1 -- 172.21.15.64:0/3379172970 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc01b050 con 0x5642e6f8db20 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.626+0000 7f41c27fc640 1 -- 172.21.15.64:0/3379172970 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc014de0 con 0x5642e6f8db20 2024-04-01T00:14:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3379172970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 msgr2=0x5642e6f8df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00fc00 tx=0x7f41cc012000 comp rx=0 tx=0).stop 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3379172970 shutdown_connections 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6f604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6fae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3379172970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6f8db20 0x5642e6f8df00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3379172970 >> 172.21.15.64:0/3379172970 conn(0x5642e6f62750 msgr2=0x5642e6f64010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3379172970 shutdown_connections 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3379172970 wait complete. 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.627+0000 7f41e1d8f740 1 -- start start 2024-04-01T00:14:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 Processor -- start 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6fb70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5642e6f90e70 con 0x5642e6fc2ac0 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5642e6e85f70 con 0x5642e6fadef0 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.628+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5642e6fac890 con 0x5642e6fad610 2024-04-01T00:14:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41e1d8f740 10 client.0 _ll_register_callbacks cb 0x5642e6bddac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6fb70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38700/0 (socket says 172.21.15.64:38700) 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:48840/0 (socket says 172.21.15.64:48840) 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 -- 172.21.15.64:0/3564634304 learned_addr learned my addr 172.21.15.64:0/3564634304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 -- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 msgr2=0x5642e6fb70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6fb70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 -- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 msgr2=0x5642e6f7b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 -- 172.21.15.64:0/3564634304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e6ca86b0 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41d7fff640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41dd749640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41d0007080 tx=0x7f41d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bab8f660a0010e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0039040 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f41d0034d00 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41cc01b050 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f41d0052050 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41e1d8f740 10 client.5388 fetch_fsmap learned FSMap version 14 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.629+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5642e6fb5860 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc01b050 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f41d0055020 con 0x5642e6fc2ac0 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 10 client.5388 fetch_fsmap finished waiting for FSMap version 14 2024-04-01T00:14:54.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 10 client.5388 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 20 client.5388 populate_metadata read hostname 'smithi064' 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6fb5420 0x5642e6f91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 4 client.5388 mds_command: new command op to 5224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 --> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e6ed7030 con 0x5642e6fb5420 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41dcf48640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6fb5420 0x5642e6f91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41b77fe640 10 client.5388 ms_handle_connect on v2:172.21.15.100:6835/1975239378 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.630+0000 7f41dcf48640 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6fb5420 0x5642e6f91980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cd3a423c265bf5 server_cookie=a17f5ff55ceeceaa in_seq=0 out_seq=0 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41b77fe640 1 -- 172.21.15.64:0/3564634304 <== mds.0 v2:172.21.15.100:6835/1975239378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5642e6ed7030 con 0x5642e6fb5420 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41b77fe640 10 client.5388 handle_command_reply: tid=0 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 client.5388 shutdown 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 msgr2=0x5642e6fb4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41d0007080 tx=0x7f41d0004ab0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 shutdown_connections 2024-04-01T00:14:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6fb5420 0x5642e6f91980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fc2ac0 0x5642e6fb4d60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/3564634304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fad610 0x5642e6fb70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41b77fe640 0 client.5388 ms_handle_reset on v2:172.21.15.100:6835/1975239378 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 >> 172.21.15.64:0/3564634304 conn(0x5642e6f8db20 msgr2=0x5642e6f64720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 shutdown_connections 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 1 -- 172.21.15.64:0/3564634304 wait complete. 2024-04-01T00:14:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.631+0000 7f41e1d8f740 20 client.5388 trim_cache size 0 max 16384 2024-04-01T00:14:54.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.649+0000 7f41e1d8f740 1 Processor -- start 2024-04-01T00:14:54.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.649+0000 7f41e1d8f740 1 -- start start 2024-04-01T00:14:54.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6fb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8169bb0 con 0x5642e6fad610 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80c7ef0 con 0x5642e6fadef0 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f41d8169560 con 0x5642e6fc2ac0 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38716/0 (socket says 172.21.15.64:38716) 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 -- 172.21.15.64:0/1008736382 learned_addr learned my addr 172.21.15.64:0/1008736382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dd749640 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6fb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41d7fff640 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 -- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 msgr2=0x5642e6fb7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6fb7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 -- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 msgr2=0x5642e6f7af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.650+0000 7f41dcf48640 1 -- 172.21.15.64:0/1008736382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e6ec5d10 con 0x5642e6fad610 2024-04-01T00:14:54.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41dcf48640 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41cc020000 tx=0x7f41cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362cabb784c94469 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41c1ffb640 1 -- 172.21.15.64:0/1008736382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc025040 con 0x5642e6fad610 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41c1ffb640 1 -- 172.21.15.64:0/1008736382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc01b040 con 0x5642e6fad610 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41c1ffb640 1 -- 172.21.15.64:0/1008736382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc033520 con 0x5642e6fad610 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 -- 172.21.15.64:0/1008736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 msgr2=0x5642e6f939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41cc020000 tx=0x7f41cc015000 comp rx=0 tx=0).stop 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 -- 172.21.15.64:0/1008736382 shutdown_connections 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6fb7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f7af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/1008736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6f939a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 -- 172.21.15.64:0/1008736382 >> 172.21.15.64:0/1008736382 conn(0x5642e6f8db20 msgr2=0x5642e6f5b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 -- 172.21.15.64:0/1008736382 shutdown_connections 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.651+0000 7f41e1d8f740 1 -- 172.21.15.64:0/1008736382 wait complete. 2024-04-01T00:14:54.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.652+0000 7f41e1d8f740 1 -- start start 2024-04-01T00:14:54.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.652+0000 7f41e1d8f740 1 Processor -- start 2024-04-01T00:14:54.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6f64010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5642e6f65df0 con 0x5642e6fad610 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5642e6fb52a0 con 0x5642e6fadef0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5642e6f66440 con 0x5642e6fc2ac0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 10 client.0 _ll_register_callbacks cb 0x7f41d0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38730/0 (socket says 172.21.15.64:38730) 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 -- 172.21.15.64:0/756961383 learned_addr learned my addr 172.21.15.64:0/756961383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41d7fff640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dd749640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6f64010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 -- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 msgr2=0x5642e6f64010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6f64010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 -- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 msgr2=0x5642e6f92050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f92050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 -- 172.21.15.64:0/756961383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e6faed30 con 0x5642e6fad610 2024-04-01T00:14:54.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41dcf48640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41cc033040 tx=0x7f41cc001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2d510e49973dab server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc038020 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f41cc007440 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.653+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41cc01b040 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f41cc03f050 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 10 client.5394 fetch_fsmap learned FSMap version 14 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5642e6faea20 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc01b040 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f41cc042020 con 0x5642e6fad610 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 10 client.5394 fetch_fsmap finished waiting for FSMap version 14 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 10 client.5394 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 20 client.5394 populate_metadata read hostname 'smithi064' 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6f64550 0x5642e6f5f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 4 client.5394 mds_command: new command op to 5224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 --> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] -- command(tid 0: {"prefix": "status"}) -- 0x5642e6ed1380 con 0x5642e6f64550 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41d7fff640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6f64550 0x5642e6f5f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41b77fe640 10 client.5394 ms_handle_connect on v2:172.21.15.100:6835/1975239378 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.654+0000 7f41d7fff640 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6f64550 0x5642e6f5f2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694d3bec75ff0c1f server_cookie=2e79ec3b53216eac in_seq=0 out_seq=0 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41b77fe640 1 -- 172.21.15.64:0/756961383 <== mds.0 v2:172.21.15.100:6835/1975239378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5642e6ed1380 con 0x5642e6f64550 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41b77fe640 10 client.5394 handle_command_reply: tid=0 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 client.5394 shutdown 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 msgr2=0x5642e6fb3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41cc033040 tx=0x7f41cc001080 comp rx=0 tx=0).stop 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 shutdown_connections 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:6835/1975239378,v1:172.21.15.100:6837/1975239378] conn(0x5642e6f64550 0x5642e6f5f2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5642e6fc2ac0 0x5642e6f64010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642e6fadef0 0x5642e6f92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41b77fe640 0 client.5394 ms_handle_reset on v2:172.21.15.100:6835/1975239378 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 --2- 172.21.15.64:0/756961383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642e6fad610 0x5642e6fb3b00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 >> 172.21.15.64:0/756961383 conn(0x5642e6f8db20 msgr2=0x5642e6f8b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 shutdown_connections 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 1 -- 172.21.15.64:0/756961383 wait complete. 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41e1d8f740 20 client.5394 trim_cache size 0 max 16384 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "05d5f08b-c3c0-4c90-b927-5e55cbe6aed5", 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5224, 2024-04-01T00:14:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.0818737209999991, 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 14, 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 33, 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 33, 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 15.969610565 2024-04-01T00:14:54.664 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 msgr2=0x7f41bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 0x7f41bc06de50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000f60 tx=0x7f41cc009040 comp rx=0 tx=0).stop 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 msgr2=0x7f41d8143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.655+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c530 tx=0x7f41c400c9f0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 shutdown_connections 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f41bc06b990 0x7f41bc06de50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d80cd660 0x7f41d8149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d80c6500 0x7f41d8143f80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 --2- 172.21.15.64:0/2484653283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d80c5c50 0x7f41d8072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 >> 172.21.15.64:0/2484653283 conn(0x7f41d80cfa90 msgr2=0x7f41d80af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 shutdown_connections 2024-04-01T00:14:54.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.656+0000 7f41df1ac640 1 -- 172.21.15.64:0/2484653283 wait complete. 2024-04-01T00:14:54.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-01T00:14:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.767+0000 7f42827c7640 1 Processor -- start 2024-04-01T00:14:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.767+0000 7f42827c7640 1 -- start start 2024-04-01T00:14:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f42827c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f42827c7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f427c0b3820 0x7f427c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f42827c7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427c0b3c80 con 0x7f427c0cf910 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f42827c7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f427c0d2ba0 con 0x7f427c0b3820 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f42827c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f427c0d2e80 con 0x7f427c1619e0 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f427bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f427bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:53540/0 (socket says 172.21.15.64:53540) 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f427bfff640 1 -- 172.21.15.64:0/380776540 learned_addr learned my addr 172.21.15.64:0/380776540 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.768+0000 7f427b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f427c0b3820 0x7f427c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.769+0000 7f427bfff640 1 -- 172.21.15.64:0/380776540 >> v2:172.21.15.100:3300/0 conn(0x7f427c0b3820 msgr2=0x7f427c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.769+0000 7f427bfff640 1 --2- 172.21.15.64:0/380776540 >> v2:172.21.15.100:3300/0 conn(0x7f427c0b3820 0x7f427c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.769+0000 7f427bfff640 1 -- 172.21.15.64:0/380776540 >> v1:172.21.15.100:6789/0 conn(0x7f427c0cf910 legacy=0x7f427c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.769+0000 7f427bfff640 1 -- 172.21.15.64:0/380776540 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c0ceeb0 con 0x7f427c1619e0 2024-04-01T00:14:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.769+0000 7f427bfff640 1 --2- 172.21.15.64:0/380776540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb44483d32b7799f server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f427affd640 1 -- 172.21.15.64:0/380776540 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c035020 con 0x7f427c1619e0 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f427affd640 1 -- 172.21.15.64:0/380776540 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c031640 con 0x7f427c1619e0 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f427affd640 1 -- 172.21.15.64:0/380776540 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c02d040 con 0x7f427c1619e0 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 -- 172.21.15.64:0/380776540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 msgr2=0x7f427c0d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 --2- 172.21.15.64:0/380776540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c02b8f0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 -- 172.21.15.64:0/380776540 shutdown_connections 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 --2- 172.21.15.64:0/380776540 >> v2:172.21.15.100:3300/0 conn(0x7f427c0b3820 0x7f427c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 --2- 172.21.15.64:0/380776540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c1619e0 0x7f427c0d4a60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 -- 172.21.15.64:0/380776540 >> 172.21.15.64:0/380776540 conn(0x7f427c09d290 msgr2=0x7f427c0d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 -- 172.21.15.64:0/380776540 shutdown_connections 2024-04-01T00:14:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.770+0000 7f42827c7640 1 -- 172.21.15.64:0/380776540 wait complete. 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.771+0000 7f42827c7640 1 Processor -- start 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 -- start start 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 0x7f427c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b31b0 con 0x7f427c1619e0 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0d2890 con 0x7f427c0b3820 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f42827c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f427c0ceb10 con 0x7f427c0cf910 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f4280d64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f427bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 0x7f427c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f4280d64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38740/0 (socket says 172.21.15.64:38740) 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f427bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:48882/0 (socket says 172.21.15.64:48882) 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.772+0000 7f4280d64640 1 -- 172.21.15.64:0/1315409857 learned_addr learned my addr 172.21.15.64:0/1315409857 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 -- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 msgr2=0x7f427c152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 0x7f427c152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 -- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 msgr2=0x7f427c1f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 -- 172.21.15.64:0/1315409857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c152640 con 0x7f427c1619e0 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f427b7fe640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 0x7f427c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f427bfff640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4280d64640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f427400c710 tx=0x7f427400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274016020 con 0x7f427c1619e0 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4274006d50 con 0x7f427c1619e0 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c14f940 con 0x7f427c1619e0 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274003bb0 con 0x7f427c1619e0 2024-04-01T00:14:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.773+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1f79f0 con 0x7f427c1619e0 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.774+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c0681a0 con 0x7f427c1619e0 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.775+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f427400a850 con 0x7f427c1619e0 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.775+0000 7f4278ff9640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 0x7f426806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.775+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f427408a340 con 0x7f427c1619e0 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.775+0000 7f427bfff640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 0x7f426806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:14:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.776+0000 7f427bfff640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 0x7f426806de50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f426c036000 tx=0x7f426c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:14:54.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:54.779+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4274020020 con 0x7f427c1619e0 2024-04-01T00:14:55.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.006+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f427c09d730 con 0x7f427c1619e0 2024-04-01T00:14:55.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.006+0000 7f4278ff9640 1 -- 172.21.15.64:0/1315409857 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f427c14f940 con 0x7f427c1619e0 2024-04-01T00:14:55.008 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 msgr2=0x7f426806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 0x7f426806de50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f426c036000 tx=0x7f426c03c040 comp rx=0 tx=0).stop 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 msgr2=0x7f427c14f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f427400c710 tx=0x7f427400cbd0 comp rx=0 tx=0).stop 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 shutdown_connections 2024-04-01T00:14:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f426806b990 0x7f426806de50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f427c1619e0 0x7f427c14f400 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f427c0cf910 0x7f427c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 --2- 172.21.15.64:0/1315409857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f427c0b3820 0x7f427c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 >> 172.21.15.64:0/1315409857 conn(0x7f427c09d290 msgr2=0x7f427c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 shutdown_connections 2024-04-01T00:14:55.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55.008+0000 7f42827c7640 1 -- 172.21.15.64:0/1315409857 wait complete. 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-01T00:14:55.021 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-01T00:14:55.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.022 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.037 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42697sec preferred_lft 42697sec 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-04-01T00:14:55.038 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether a6:ca:d0:36:05:dc brd ff:ff:ff:ff:ff:ff 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e041:2bff:fe60:1b0c/64 scope link 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:26:c8:bb:aa:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::f826:c8ff:febb:aa50/64 scope link 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether a6:ca:d0:36:05:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::a4ca:d0ff:fe36:5dc/64 scope link 2024-04-01T00:14:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-04-01T00:14:55.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.040 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-01T00:14:55.103 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-04-01T00:14:55.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.104 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-04-01T00:14:55.159 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-04-01T00:14:55.160 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-01T00:14:55.160 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-01T00:14:55.160 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-01T00:14:55.160 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-01T00:14:55.160 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-01T00:14:55.160 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-04-01T00:14:55.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-01T00:14:55.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-01T00:14:55.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-01T00:14:55.161 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-04-01T00:14:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-01T00:14:55.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:55.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.295 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-04-01T00:14:55.309 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-01T00:14:55.309 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-01T00:14:55.310 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-01T00:14:55.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-04-01T00:14:55.379 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-01T00:14:55.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:55.379 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:55.434 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-01T00:14:55.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-01T00:14:55.434 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:55.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:55.557 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:55.612 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:55.613 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:55.614 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:55.615 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:55.615 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:55.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.615 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:55.667 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-01T00:14:55.667 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-01T00:14:55.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-04-01T00:14:55.736 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-04-01T00:14:55.792 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-04-01T00:14:55.793 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:55.794 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T00:14:55.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.795 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-04-01T00:14:55.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-04-01T00:14:55.793+0000 7f2b7a43a480 -1 init, newargv = 0x55ee9ef04c50 newargc=15 2024-04-01T00:14:55.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52676]: starting ceph client 2024-04-01T00:14:55.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52676]: starting fuse 2024-04-01T00:14:55.810 INFO:teuthology.orchestra.run.smithi064.stdout:41 2024-04-01T00:14:55.810 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-01T00:14:55.811 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-04-01T00:14:55.812 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-04-01T00:14:55.813 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-01T00:14:55.813 DEBUG:teuthology.orchestra.run.smithi064:> 2024-04-01T00:14:55.813 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-04-01T00:14:55.813 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-04-01T00:14:55.931 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.52676.asok 2024-04-01T00:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:14:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-01T00:14:55.942 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52676.asok 2024-04-01T00:14:55.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:55.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52676.asok status 2024-04-01T00:14:56.079 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T00:14:56.079 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-04-01T00:14:56.079 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "384f31f3d3aee7f8ba3f60846b8844c049165583", 2024-04-01T00:14:56.079 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-1977-g384f31f3 (384f31f3d3aee7f8ba3f60846b8844c049165583) squid (dev)", 2024-04-01T00:14:56.079 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "52676", 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5400, 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5400 2024-04-01T00:14:56.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3803293045 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3803293045 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5400 192.168.0.1:0/3803293045", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3803293045", 2024-04-01T00:14:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 14, 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 33, 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-04-01T00:14:56.082 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T00:14:56.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:56.088 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-04-01T00:14:56.088 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:56.103 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-04-01T00:14:56.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:56.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:56.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-01T00:14:56.171 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=xfstests-dev 2024-04-01T00:14:56.226 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.ALJJcUYdNQxfstests-dev 2024-04-01T00:14:56.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:56.227 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.ALJJcUYdNQxfstests-dev) 2024-04-01T00:14:56.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:56.296 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.ALJJcUYdNQxfstests-dev) 2024-04-01T00:14:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:Cloning into '/tmp/tmp.ALJJcUYdNQxfstests-dev'... 2024-04-01T00:14:57.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:57.106 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-01T00:14:57.167 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=test 2024-04-01T00:14:57.222 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.vihoW6DmoUtest 2024-04-01T00:14:57.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:14:57.222 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-01T00:14:57.280 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=scratch 2024-04-01T00:14:57.334 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.uZA5HvZYq0scratch 2024-04-01T00:14:57.334 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-01T00:14:57.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-01T00:14:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:03:37 ago on Mon 01 Apr 2024 12:11:20 AM UTC. 2024-04-01T00:14:58.052 INFO:teuthology.orchestra.run.smithi064.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-01T00:14:58.053 INFO:teuthology.orchestra.run.smithi064.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-01T00:14:58.054 INFO:teuthology.orchestra.run.smithi064.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-01T00:14:58.055 INFO:teuthology.orchestra.run.smithi064.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-01T00:14:58.056 INFO:teuthology.orchestra.run.smithi064.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-01T00:14:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-04-01T00:14:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-01T00:14:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-04-01T00:14:58.063 INFO:teuthology.orchestra.run.smithi064.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-04-01T00:14:58.067 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-01T00:14:58.068 INFO:teuthology.orchestra.run.smithi064.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-01T00:14:58.070 INFO:teuthology.orchestra.run.smithi064.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-01T00:14:58.071 INFO:teuthology.orchestra.run.smithi064.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-01T00:14:58.072 INFO:teuthology.orchestra.run.smithi064.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-04-01T00:14:58.073 INFO:teuthology.orchestra.run.smithi064.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-04-01T00:14:58.074 INFO:teuthology.orchestra.run.smithi064.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-01T00:14:58.076 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-04-01T00:14:58.076 INFO:teuthology.orchestra.run.smithi064.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-01T00:14:58.078 INFO:teuthology.orchestra.run.smithi064.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-01T00:14:58.081 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-01T00:14:58.149 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-devel x86_64 5.14.0-434.el9 CentOS-AppStream 16 M 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-04-01T00:14:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: cpp x86_64 11.4.1-3.el9 appstream 11 M 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper x86_64 9:1.02.197-2.el9 baseos 141 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 baseos 34 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 32 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 baseos 179 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: e2fsprogs x86_64 1.46.5-5.el9 baseos 1.0 M 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 baseos 222 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: gcc x86_64 11.4.1-3.el9 appstream 32 M 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: gcc-c++ x86_64 11.4.1-3.el9 appstream 13 M 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 appstream 47 k 2024-04-01T00:14:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: git x86_64 2.43.0-1.el9 appstream 51 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: git-core x86_64 2.43.0-1.el9 appstream 4.4 M 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: git-core-doc noarch 2.43.0-1.el9 appstream 2.9 M 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err x86_64 1.46.5-5.el9 baseos 26 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libgcc x86_64 11.4.1-3.el9 baseos 92 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libgomp x86_64 11.4.1-3.el9 baseos 268 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libss x86_64 1.46.5-5.el9 baseos 32 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libstdc++ x86_64 11.4.1-3.el9 baseos 743 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: libstdc++-devel x86_64 11.4.1-3.el9 appstream 2.4 M 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: make x86_64 1:4.3-8.el9 baseos 536 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: perl-Git noarch 2.43.0-1.el9 appstream 38 k 2024-04-01T00:14:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-04-01T00:14:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout:Install 17 Packages 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 28 Packages 2024-04-01T00:14:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:14:58.158 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 103 M 2024-04-01T00:14:58.158 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-04-01T00:14:59.018 INFO:teuthology.orchestra.run.smithi064.stdout:(1/45): fio-engine-http-3.35-1.el9.x86_64.rpm 100 kB/s | 17 kB 00:00 2024-04-01T00:14:59.034 INFO:teuthology.orchestra.run.smithi064.stdout:(2/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 77 kB/s | 14 kB 00:00 2024-04-01T00:14:59.185 INFO:teuthology.orchestra.run.smithi064.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 81 kB/s | 13 kB 00:00 2024-04-01T00:14:59.210 INFO:teuthology.orchestra.run.smithi064.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 83 kB/s | 15 kB 00:00 2024-04-01T00:14:59.344 INFO:teuthology.orchestra.run.smithi064.stdout:(5/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 114 kB/s | 15 kB 00:00 2024-04-01T00:14:59.361 INFO:teuthology.orchestra.run.smithi064.stdout:(6/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 130 kB/s | 20 kB 00:00 2024-04-01T00:14:59.444 INFO:teuthology.orchestra.run.smithi064.stdout:(7/45): fio-3.35-1.el9.x86_64.rpm 10 MB/s | 6.2 MB 00:00 2024-04-01T00:14:59.461 INFO:teuthology.orchestra.run.smithi064.stdout:(8/45): libcap-devel-2.48-9.el9.x86_64.rpm 331 kB/s | 33 kB 00:00 2024-04-01T00:14:59.795 INFO:teuthology.orchestra.run.smithi064.stdout:(9/45): kernel-devel-5.14.0-434.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-01T00:14:59.828 INFO:teuthology.orchestra.run.smithi064.stdout:(10/45): sqlite-3.34.1-7.el9.x86_64.rpm 1.9 MB/s | 747 kB 00:00 2024-04-01T00:14:59.853 INFO:teuthology.orchestra.run.smithi064.stdout:(11/45): udftools-2.2-5.el9.x86_64.rpm 368 kB/s | 144 kB 00:00 2024-04-01T00:14:59.879 INFO:teuthology.orchestra.run.smithi064.stdout:(12/45): rmt-1.6-6.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-04-01T00:15:00.063 INFO:teuthology.orchestra.run.smithi064.stdout:(13/45): gdbm-devel-1.19-4.el9.x86_64.rpm 320 kB/s | 59 kB 00:00 2024-04-01T00:15:00.105 INFO:teuthology.orchestra.run.smithi064.stdout:(14/45): libnbd-1.18.1-3.el9.x86_64.rpm 664 kB/s | 166 kB 00:00 2024-04-01T00:15:00.130 INFO:teuthology.orchestra.run.smithi064.stdout:(15/45): gdbm-1.19-4.el9.x86_64.rpm 480 kB/s | 132 kB 00:00 2024-04-01T00:15:00.169 INFO:teuthology.orchestra.run.smithi064.stdout:(16/45): python-unversioned-command-3.9.18-3.el 215 kB/s | 8.3 kB 00:00 2024-04-01T00:15:00.197 INFO:teuthology.orchestra.run.smithi064.stdout:(17/45): dump-0.4-0.51.b47.el9.x86_64.rpm 653 kB/s | 88 kB 00:00 2024-04-01T00:15:00.223 INFO:teuthology.orchestra.run.smithi064.stdout:(18/45): python3-devel-3.9.18-3.el9.x86_64.rpm 3.7 MB/s | 204 kB 00:00 2024-04-01T00:15:00.248 INFO:teuthology.orchestra.run.smithi064.stdout:(19/45): python3-3.9.18-3.el9.x86_64.rpm 504 kB/s | 25 kB 00:00 2024-04-01T00:15:00.275 INFO:teuthology.orchestra.run.smithi064.stdout:(20/45): indent-2.2.13-5.el9.x86_64.rpm 1.0 MB/s | 179 kB 00:00 2024-04-01T00:15:00.423 INFO:teuthology.orchestra.run.smithi064.stdout:(21/45): python3-libs-3.9.18-3.el9.x86_64.rpm 36 MB/s | 7.3 MB 00:00 2024-04-01T00:15:00.449 INFO:teuthology.orchestra.run.smithi064.stdout:(22/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 3.0 MB/s | 619 kB 00:00 2024-04-01T00:15:01.032 INFO:teuthology.orchestra.run.smithi064.stdout:(23/45): device-mapper-event-1.02.197-2.el9.x86 56 kB/s | 34 kB 00:00 2024-04-01T00:15:01.057 INFO:teuthology.orchestra.run.smithi064.stdout:(24/45): device-mapper-event-libs-1.02.197-2.el 53 kB/s | 32 kB 00:00 2024-04-01T00:15:01.133 INFO:teuthology.orchestra.run.smithi064.stdout:(25/45): device-mapper-1.02.197-2.el9.x86_64.rp 165 kB/s | 141 kB 00:00 2024-04-01T00:15:01.892 INFO:teuthology.orchestra.run.smithi064.stdout:(26/45): device-mapper-libs-1.02.197-2.el9.x86_ 208 kB/s | 179 kB 00:00 2024-04-01T00:15:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:(27/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 251 kB/s | 222 kB 00:00 2024-04-01T00:15:02.467 INFO:teuthology.orchestra.run.smithi064.stdout:(28/45): libcom_err-1.46.5-5.el9.x86_64.rpm 46 kB/s | 26 kB 00:00 2024-04-01T00:15:02.759 INFO:teuthology.orchestra.run.smithi064.stdout:(29/45): libgcc-11.4.1-3.el9.x86_64.rpm 125 kB/s | 92 kB 00:00 2024-04-01T00:15:02.993 INFO:teuthology.orchestra.run.smithi064.stdout:(30/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 531 kB/s | 1.0 MB 00:01 2024-04-01T00:15:03.376 INFO:teuthology.orchestra.run.smithi064.stdout:(31/45): libss-1.46.5-5.el9.x86_64.rpm 52 kB/s | 32 kB 00:00 2024-04-01T00:15:03.401 INFO:teuthology.orchestra.run.smithi064.stdout:(32/45): libgomp-11.4.1-3.el9.x86_64.rpm 287 kB/s | 268 kB 00:00 2024-04-01T00:15:04.593 INFO:teuthology.orchestra.run.smithi064.stdout:(33/45): libstdc++-11.4.1-3.el9.x86_64.rpm 472 kB/s | 743 kB 00:01 2024-04-01T00:15:05.411 INFO:teuthology.orchestra.run.smithi064.stdout:(34/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 509 kB/s | 1.0 MB 00:02 2024-04-01T00:15:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:(35/45): cpp-11.4.1-3.el9.x86_64.rpm 23 MB/s | 11 MB 00:00 2024-04-01T00:15:05.903 INFO:teuthology.orchestra.run.smithi064.stdout:(36/45): make-4.3-8.el9.x86_64.rpm 410 kB/s | 536 kB 00:01 2024-04-01T00:15:06.120 INFO:teuthology.orchestra.run.smithi064.stdout:(37/45): lvm2-2.03.23-2.el9.x86_64.rpm 573 kB/s | 1.5 MB 00:02 2024-04-01T00:15:06.328 INFO:teuthology.orchestra.run.smithi064.stdout:(38/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 227 kB/s | 47 kB 00:00 2024-04-01T00:15:06.636 INFO:teuthology.orchestra.run.smithi064.stdout:(39/45): gcc-11.4.1-3.el9.x86_64.rpm 42 MB/s | 32 MB 00:00 2024-04-01T00:15:06.678 INFO:teuthology.orchestra.run.smithi064.stdout:(40/45): git-2.43.0-1.el9.x86_64.rpm 145 kB/s | 51 kB 00:00 2024-04-01T00:15:06.804 INFO:teuthology.orchestra.run.smithi064.stdout:(41/45): git-core-doc-2.43.0-1.el9.noarch.rpm 23 MB/s | 2.9 MB 00:00 2024-04-01T00:15:06.912 INFO:teuthology.orchestra.run.smithi064.stdout:(42/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 22 MB/s | 2.4 MB 00:00 2024-04-01T00:15:06.971 INFO:teuthology.orchestra.run.smithi064.stdout:(43/45): perl-Git-2.43.0-1.el9.noarch.rpm 651 kB/s | 38 kB 00:00 2024-04-01T00:15:07.163 INFO:teuthology.orchestra.run.smithi064.stdout:(44/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 10 MB/s | 13 MB 00:01 2024-04-01T00:15:07.230 INFO:teuthology.orchestra.run.smithi064.stdout:(45/45): git-core-2.43.0-1.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-04-01T00:15:07.234 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-04-01T00:15:07.234 INFO:teuthology.orchestra.run.smithi064.stdout:Total 11 MB/s | 103 MB 00:09 2024-04-01T00:15:08.072 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-04-01T00:15:08.392 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-04-01T00:15:08.392 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-04-01T00:15:09.704 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-04-01T00:15:09.704 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-04-01T00:15:12.750 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-04-01T00:15:12.814 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-04-01T00:15:12.877 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-04-01T00:15:12.966 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-04-01T00:15:13.023 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-04-01T00:15:13.078 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-04-01T00:15:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-04-01T00:15:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-04-01T00:15:13.404 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-04-01T00:15:13.600 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 8/73 2024-04-01T00:15:13.660 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : make-1:4.3-8.el9.x86_64 9/73 2024-04-01T00:15:14.992 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 10/73 2024-04-01T00:15:15.113 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 11/73 2024-04-01T00:15:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 12/73 2024-04-01T00:15:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 13/73 2024-04-01T00:15:16.082 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-04-01T00:15:16.122 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-04-01T00:15:16.159 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-04-01T00:15:16.191 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-04-01T00:15:16.428 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-04-01T00:15:16.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:15:16.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:15:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-04-01T00:15:17.182 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-04-01T00:15:17.527 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-04-01T00:15:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 21/73 2024-04-01T00:15:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 22/73 2024-04-01T00:15:18.710 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-04-01T00:15:18.751 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-04-01T00:15:18.786 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-04-01T00:15:18.813 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 25/73 2024-04-01T00:15:18.838 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 26/73 2024-04-01T00:15:18.864 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 27/73 2024-04-01T00:15:18.889 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 28/73 2024-04-01T00:15:18.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 29/73 2024-04-01T00:15:18.940 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 30/73 2024-04-01T00:15:19.102 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 31/73 2024-04-01T00:15:19.196 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fio-3.35-1.el9.x86_64 32/73 2024-04-01T00:15:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rmt-2:1.6-6.el9.x86_64 33/73 2024-04-01T00:15:19.262 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-04-01T00:15:29.181 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-04-01T00:15:29.364 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-devel-5.14.0-434.el9.x86_64 36/73 2024-04-01T00:15:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-devel-5.14.0-434.el9.x86_64 36/73 2024-04-01T00:15:39.820 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-04-01T00:15:39.852 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-04-01T00:15:39.946 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-04-01T00:15:40.110 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-04-01T00:15:40.110 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-04-01T00:15:40.111 INFO:teuthology.orchestra.run.smithi064.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-04-01T00:15:40.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:15:40.406 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-04-01T00:15:40.451 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-04-01T00:15:40.518 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-04-01T00:15:40.555 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-04-01T00:15:40.605 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-04-01T00:15:40.630 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-04-01T00:15:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-04-01T00:15:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 47/73 2024-04-01T00:15:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 48/73 2024-04-01T00:15:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 49/73 2024-04-01T00:15:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-04-01T00:15:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-04-01T00:15:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-04-01T00:15:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-04-01T00:15:40.886 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 53/73 2024-04-01T00:15:40.894 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-01T00:15:40.894 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-01T00:15:40.948 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-01T00:15:40.994 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 55/73 2024-04-01T00:15:41.029 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 56/73 2024-04-01T00:15:41.054 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 57/73 2024-04-01T00:15:41.079 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 58/73 2024-04-01T00:15:41.084 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-04-01T00:15:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-04-01T00:15:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 60/73 2024-04-01T00:15:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 61/73 2024-04-01T00:15:41.189 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-04-01T00:15:41.214 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-04-01T00:15:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-04-01T00:15:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-04-01T00:15:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-04-01T00:15:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-04-01T00:15:41.418 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-04-01T00:15:41.418 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-04-01T00:15:41.507 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-04-01T00:15:41.538 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 70/73 2024-04-01T00:15:41.562 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 71/73 2024-04-01T00:15:41.588 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 72/73 2024-04-01T00:15:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : make-1:4.3-7.el9.x86_64 73/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: make-1:4.3-7.el9.x86_64 73/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-devel-5.14.0-434.el9.x86_64 8/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 9/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 10/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udftools-2.2-5.el9.x86_64 11/73 2024-04-01T00:15:44.169 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 12/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 13/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 18/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 19/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 20/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 21/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-3.9.18-3.el9.x86_64 22/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-3.9.17-2.el9.x86_64 23/73 2024-04-01T00:15:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 24/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 25/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 26/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 27/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 28/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 29/73 2024-04-01T00:15:44.171 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 30/73 2024-04-01T00:15:44.175 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 31/73 2024-04-01T00:15:44.175 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 32/73 2024-04-01T00:15:44.175 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 33/73 2024-04-01T00:15:44.175 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 34/73 2024-04-01T00:15:44.175 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 35/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 36/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 37/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 38/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 39/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 40/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 41/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 42/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 43/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 44/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 45/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libss-1.46.5-5.el9.x86_64 46/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libss-1.46.5-3.el9.x86_64 47/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 48/73 2024-04-01T00:15:44.176 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 49/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 50/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 51/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 52/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : make-1:4.3-8.el9.x86_64 54/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : make-1:4.3-7.el9.x86_64 55/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 56/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 57/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 58/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 59/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 60/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 61/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 62/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 63/73 2024-04-01T00:15:44.177 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-2.43.0-1.el9.x86_64 64/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-2.39.3-1.el9.x86_64 65/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 66/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 67/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 68/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 69/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 70/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 71/73 2024-04-01T00:15:44.178 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 72/73 2024-04-01T00:15:45.192 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 73/73 2024-04-01T00:15:45.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:15:45.192 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-04-01T00:15:45.192 INFO:teuthology.orchestra.run.smithi064.stdout: cpp-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: gcc-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: git-2.43.0-1.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: git-core-2.43.0-1.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: libgcc-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.193 INFO:teuthology.orchestra.run.smithi064.stdout: libgomp-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: libss-1.46.5-5.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: make-1:4.3-8.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: perl-Git-2.43.0-1.el9.noarch 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: python3-3.9.18-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-04-01T00:15:45.194 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-434.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: udftools-2.2-5.el9.x86_64 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:15:45.195 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-04-01T00:15:45.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo useradd -m fsgqa 2024-04-01T00:15:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:useradd: user 'fsgqa' already exists 2024-04-01T00:15:45.976 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-01T00:15:45.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo groupadd fsgqa 2024-04-01T00:15:46.044 INFO:teuthology.orchestra.run.smithi064.stderr:groupadd: group 'fsgqa' already exists 2024-04-01T00:15:46.045 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-01T00:15:46.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo useradd fsgqa2 2024-04-01T00:15:46.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:15:46.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:15:46.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo useradd 123456-fsgqa 2024-04-01T00:15:47.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-01T00:15:47.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.505+0000 7ff494ad4640 1 Processor -- start 2024-04-01T00:15:47.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.505+0000 7ff494ad4640 1 -- start start 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff494ad4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff4900d4a60 0x7ff4900b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff494ad4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900d2ba0 con 0x7ff4901619e0 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff494ad4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900d2e80 con 0x7ff4900cfae0 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff494ad4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900ceeb0 con 0x7ff4900d4a60 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff48e59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff4901619e0 0x7ff4900cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59880/0 (socket says 172.21.15.64:59880) 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff48e59c640 1 -- 172.21.15.64:0/2960352492 learned_addr learned my addr 172.21.15.64:0/2960352492 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.506+0000 7ff48dd9b640 1 --2- 172.21.15.64:0/2960352492 >> v2:172.21.15.100:3300/0 conn(0x7ff4900d4a60 0x7ff4900b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129271166 0 0) 0x7ff4900d2ba0 con 0x7ff4901619e0 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff47c003520 con 0x7ff4901619e0 2024-04-01T00:15:47.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268920967 0 0) 0x7ff4900d2e80 con 0x7ff4900cfae0 2024-04-01T00:15:47.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4900d2ba0 con 0x7ff4900cfae0 2024-04-01T00:15:47.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48dd9b640 1 -- 172.21.15.64:0/2960352492 >> v1:172.21.15.100:6789/0 conn(0x7ff4900cfae0 legacy=0x7ff4901620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48dd9b640 1 -- 172.21.15.64:0/2960352492 >> v1:172.21.15.64:6789/0 conn(0x7ff4901619e0 legacy=0x7ff4900cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.507+0000 7ff48dd9b640 1 -- 172.21.15.64:0/2960352492 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff490161dc0 con 0x7ff4900d4a60 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff48dd9b640 1 --2- 172.21.15.64:0/2960352492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4900d4a60 0x7ff4900b3820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4780097a0 tx=0x7ff47802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20b6ceeb00f4271 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478035020 con 0x7ff4900d4a60 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff478031830 con 0x7ff4900d4a60 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff48d59a640 1 -- 172.21.15.64:0/2960352492 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478031c00 con 0x7ff4900d4a60 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 -- 172.21.15.64:0/2960352492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4900d4a60 msgr2=0x7ff4900b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 --2- 172.21.15.64:0/2960352492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4900d4a60 0x7ff4900b3820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4780097a0 tx=0x7ff47802b8d0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 -- 172.21.15.64:0/2960352492 shutdown_connections 2024-04-01T00:15:47.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 --2- 172.21.15.64:0/2960352492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4900d4a60 0x7ff4900b3820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 -- 172.21.15.64:0/2960352492 >> 172.21.15.64:0/2960352492 conn(0x7ff49009d290 msgr2=0x7ff4900d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:15:47.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 -- 172.21.15.64:0/2960352492 shutdown_connections 2024-04-01T00:15:47.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.508+0000 7ff494ad4640 1 -- 172.21.15.64:0/2960352492 wait complete. 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.510+0000 7ff494ad4640 1 Processor -- start 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.510+0000 7ff494ad4640 1 -- start start 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4900d4a60 0x7ff4901f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4901619e0 0x7ff4900b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900ceb10 con 0x7ff4900cfae0 2024-04-01T00:15:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900b31b0 con 0x7ff4901619e0 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900cec90 con 0x7ff4900d4a60 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38306/0 (socket says 172.21.15.64:38306) 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 -- 172.21.15.64:0/869290620 learned_addr learned my addr 172.21.15.64:0/869290620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48dd9b640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4900d4a60 0x7ff4901f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48ed9d640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4901619e0 0x7ff4900b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 -- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4900d4a60 msgr2=0x7ff4901f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4900d4a60 0x7ff4901f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 -- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4901619e0 msgr2=0x7ff4900b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4901619e0 0x7ff4900b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 -- 172.21.15.64:0/869290620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901f4bf0 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff48e59c640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c440 tx=0x7ff48800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488017070 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488005ce0 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901f4350 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488007270 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.511+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901f4910 con 0x7ff4900cfae0 2024-04-01T00:15:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.512+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4900681a0 con 0x7ff4900cfae0 2024-04-01T00:15:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.513+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff48801a070 con 0x7ff4900cfae0 2024-04-01T00:15:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.513+0000 7ff486ffd640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 0x7ff47006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.513+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff488089a80 con 0x7ff4900cfae0 2024-04-01T00:15:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.513+0000 7ff48dd9b640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 0x7ff47006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.514+0000 7ff48dd9b640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 0x7ff47006de50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff478036000 tx=0x7ff478008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.518+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff488058050 con 0x7ff4900cfae0 2024-04-01T00:15:47.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.773+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff49009d730 con 0x7ff4900cfae0 2024-04-01T00:15:47.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.773+0000 7ff486ffd640 1 -- 172.21.15.64:0/869290620 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7ff48805c0f0 con 0x7ff4900cfae0 2024-04-01T00:15:47.776 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:15:47.776 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","modified":"2024-04-01T00:12:48.105948Z","created":"2024-04-01T00:12:48.105948Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3301","nonce":0},{"type":"v1","addr":"172.21.15.100:6790","nonce":0}]},"addr":"172.21.15.100:6790/0","public_addr":"172.21.15.100:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-01T00:15:47.776 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 1 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 msgr2=0x7ff47006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 0x7ff47006de50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff478036000 tx=0x7ff478008030 comp rx=0 tx=0).stop 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 msgr2=0x7ff4901f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c440 tx=0x7ff48800c900 comp rx=0 tx=0).stop 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 shutdown_connections 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff47006b990 0x7ff47006de50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4901619e0 0x7ff4900b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4900d4a60 0x7ff4901f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 --2- 172.21.15.64:0/869290620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4900cfae0 0x7ff4901f3c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 >> 172.21.15.64:0/869290620 conn(0x7ff49009d290 msgr2=0x7ff49006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:15:47.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 shutdown_connections 2024-04-01T00:15:47.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.776+0000 7ff494ad4640 1 -- 172.21.15.64:0/869290620 wait complete. 2024-04-01T00:15:47.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-01T00:15:47.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.890+0000 7f39c4255640 1 Processor -- start 2024-04-01T00:15:47.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.890+0000 7f39c4255640 1 -- start start 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c4255640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39bc06ed00 0x7f39bc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c4255640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39bc0cb020 con 0x7f39bc0cab60 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c4255640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39bc0cb300 con 0x7f39bc144720 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c4255640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f39bc144b00 con 0x7f39bc06ed00 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c17f0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39bc06ed00 0x7f39bc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c27f2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f39bc0cab60 0x7f39bc070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59890/0 (socket says 172.21.15.64:59890) 2024-04-01T00:15:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c27f2640 1 -- 172.21.15.64:0/3255468971 learned_addr learned my addr 172.21.15.64:0/3255468971 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T00:15:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c17f0640 1 -- 172.21.15.64:0/3255468971 >> v1:172.21.15.100:6789/0 conn(0x7f39bc144720 legacy=0x7f39bc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c17f0640 1 -- 172.21.15.64:0/3255468971 >> v1:172.21.15.64:6789/0 conn(0x7f39bc0cab60 legacy=0x7f39bc070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c17f0640 1 -- 172.21.15.64:0/3255468971 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0cb5e0 con 0x7f39bc06ed00 2024-04-01T00:15:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.891+0000 7f39c0fef640 1 -- 172.21.15.64:0/3255468971 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877477743 0 0) 0x7f39bc0cb020 con 0x7f39bc0cab60 2024-04-01T00:15:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.892+0000 7f39c17f0640 1 --2- 172.21.15.64:0/3255468971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc06ed00 0x7f39bc06ce70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39b00097a0 tx=0x7f39b002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14174061b4d4e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.892+0000 7f39c0fef640 1 -- 172.21.15.64:0/3255468971 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0034070 con 0x7f39bc06ed00 2024-04-01T00:15:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.892+0000 7f39c0fef640 1 -- 172.21.15.64:0/3255468971 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b0030610 con 0x7f39bc06ed00 2024-04-01T00:15:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.892+0000 7f39c0fef640 1 -- 172.21.15.64:0/3255468971 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00309e0 con 0x7f39bc06ed00 2024-04-01T00:15:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 -- 172.21.15.64:0/3255468971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc06ed00 msgr2=0x7f39bc06ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 --2- 172.21.15.64:0/3255468971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc06ed00 0x7f39bc06ce70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39b00097a0 tx=0x7f39b002b960 comp rx=0 tx=0).stop 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 -- 172.21.15.64:0/3255468971 shutdown_connections 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 --2- 172.21.15.64:0/3255468971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc06ed00 0x7f39bc06ce70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 -- 172.21.15.64:0/3255468971 >> 172.21.15.64:0/3255468971 conn(0x7f39bc0cfa90 msgr2=0x7f39bc0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 -- 172.21.15.64:0/3255468971 shutdown_connections 2024-04-01T00:15:47.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.893+0000 7f39c4255640 1 -- 172.21.15.64:0/3255468971 wait complete. 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.894+0000 7f39c4255640 1 Processor -- start 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.894+0000 7f39c4255640 1 -- start start 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc06ed00 0x7f39bc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc144720 0x7f39bc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd7f0 con 0x7f39bc0cab60 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cbba0 con 0x7f39bc144720 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c4255640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd970 con 0x7f39bc06ed00 2024-04-01T00:15:47.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38328/0 (socket says 172.21.15.64:38328) 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 -- 172.21.15.64:0/2994254607 learned_addr learned my addr 172.21.15.64:0/2994254607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c27f2640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc144720 0x7f39bc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c1ff1640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc06ed00 0x7f39bc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 -- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc06ed00 msgr2=0x7f39bc0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc06ed00 0x7f39bc0b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 -- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc144720 msgr2=0x7f39bc0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc144720 0x7f39bc0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.895+0000 7f39c17f0640 1 -- 172.21.15.64:0/2994254607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0c96e0 con 0x7f39bc0cab60 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39c17f0640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098c0 tx=0x7f39b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0034040 con 0x7f39bc0cab60 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b0039a10 con 0x7f39bc0cab60 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc0b2c20 con 0x7f39bc0cab60 2024-04-01T00:15:47.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0039de0 con 0x7f39bc0cab60 2024-04-01T00:15:47.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.896+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc0b34c0 con 0x7f39bc0cab60 2024-04-01T00:15:47.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.897+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc0681a0 con 0x7f39bc0cab60 2024-04-01T00:15:47.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.897+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f39b0037070 con 0x7f39bc0cab60 2024-04-01T00:15:47.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.897+0000 7f39ba7fc640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 0x7f39a806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T00:15:47.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.898+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f39b00ac870 con 0x7f39bc0cab60 2024-04-01T00:15:47.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.898+0000 7f39c1ff1640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 0x7f39a806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T00:15:47.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.898+0000 7f39c1ff1640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 0x7f39a806de50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f39b4000f60 tx=0x7f39b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T00:15:47.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:47.902+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f39b007ae30 con 0x7f39bc0cab60 2024-04-01T00:15:48.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.153+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f39bc141ad0 con 0x7f39bc0cab60 2024-04-01T00:15:48.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.153+0000 7f39ba7fc640 1 -- 172.21.15.64:0/2994254607 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v9) ==== 82+0+151 (secure 0 0 0) 0x7f39b007eed0 con 0x7f39bc0cab60 2024-04-01T00:15:48.155 INFO:teuthology.orchestra.run.smithi064.stdout:[client.admin] 2024-04-01T00:15:48.155 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAA/AlmpPqxExAA72Eit4inwHsBJbOT+zZ2jw== 2024-04-01T00:15:48.156 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow *" 2024-04-01T00:15:48.156 INFO:teuthology.orchestra.run.smithi064.stdout: caps mgr = "allow *" 2024-04-01T00:15:48.156 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow *" 2024-04-01T00:15:48.156 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow *" 2024-04-01T00:15:48.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 msgr2=0x7f39a806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:48.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 0x7f39a806de50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f39b4000f60 tx=0x7f39b4009040 comp rx=0 tx=0).stop 2024-04-01T00:15:48.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 msgr2=0x7f39bc15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098c0 tx=0x7f39b00046c0 comp rx=0 tx=0).stop 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 shutdown_connections 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39a806b990 0x7f39a806de50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc144720 0x7f39bc0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39bc0cab60 0x7f39bc15a280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 --2- 172.21.15.64:0/2994254607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc06ed00 0x7f39bc0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 >> 172.21.15.64:0/2994254607 conn(0x7f39bc0cfa90 msgr2=0x7f39bc06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 shutdown_connections 2024-04-01T00:15:48.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T00:15:48.155+0000 7f39c4255640 1 -- 172.21.15.64:0/2994254607 wait complete. 2024-04-01T00:15:48.168 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-04-01T00:15:48.183 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.84DQSDZhq8 2024-04-01T00:15:48.184 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:15:48.184 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/tmp/tmp.84DQSDZhq8 2024-04-01T00:15:48.237 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:15:48.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/tmp/tmp.ALJJcUYdNQxfstests-dev/local.config 2024-04-01T00:15:48.304 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.vihoW6DmoUtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.uZA5HvZYq0scratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.64:6789 -k /tmp/tmp.84DQSDZhq8 --client_mountpoint /test" 2024-04-01T00:15:48.304 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T00:15:48.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/tmp/tmp.ALJJcUYdNQxfstests-dev/ceph.exclude 2024-04-01T00:15:48.370 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo make) 2024-04-01T00:15:48.441 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize -cfi 2024-04-01T00:15:48.839 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-01T00:15:48.839 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file './config.guess' 2024-04-01T00:15:48.842 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file './config.sub' 2024-04-01T00:15:48.845 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file './install-sh' 2024-04-01T00:15:48.849 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file './ltmain.sh' 2024-04-01T00:15:48.870 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-01T00:15:48.870 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-01T00:15:48.875 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-01T00:15:48.878 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-01T00:15:48.883 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-01T00:15:48.887 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-01T00:15:48.893 INFO:teuthology.orchestra.run.smithi064.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-01T00:15:48.894 INFO:teuthology.orchestra.run.smithi064.stdout:cp include/install-sh . 2024-04-01T00:15:48.895 INFO:teuthology.orchestra.run.smithi064.stdout:aclocal -I m4 2024-04-01T00:15:49.919 INFO:teuthology.orchestra.run.smithi064.stdout:autoheader 2024-04-01T00:15:50.215 INFO:teuthology.orchestra.run.smithi064.stdout:autoconf 2024-04-01T00:15:50.388 INFO:teuthology.orchestra.run.smithi064.stdout:./configure \ 2024-04-01T00:15:50.389 INFO:teuthology.orchestra.run.smithi064.stdout: --libexecdir=/usr/lib \ 2024-04-01T00:15:50.389 INFO:teuthology.orchestra.run.smithi064.stdout: --exec_prefix=/var/lib 2024-04-01T00:15:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-01T00:15:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-01T00:15:50.580 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to print strings... printf 2024-04-01T00:15:50.580 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gcc... gcc 2024-04-01T00:15:50.639 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether the C compiler works... yes 2024-04-01T00:15:50.639 INFO:teuthology.orchestra.run.smithi064.stdout:checking for C compiler default output file name... a.out 2024-04-01T00:15:50.666 INFO:teuthology.orchestra.run.smithi064.stdout:checking for suffix of executables... 2024-04-01T00:15:50.720 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether we are cross compiling... no 2024-04-01T00:15:50.736 INFO:teuthology.orchestra.run.smithi064.stdout:checking for suffix of object files... o 2024-04-01T00:15:50.751 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether we are using the GNU C compiler... yes 2024-04-01T00:15:50.766 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether gcc accepts -g... yes 2024-04-01T00:15:50.792 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-01T00:15:50.794 INFO:teuthology.orchestra.run.smithi064.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-01T00:15:50.796 INFO:teuthology.orchestra.run.smithi064.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-01T00:15:50.797 INFO:teuthology.orchestra.run.smithi064.stdout:checking for egrep... /bin/grep -E 2024-04-01T00:15:50.798 INFO:teuthology.orchestra.run.smithi064.stdout:checking for fgrep... /bin/grep -F 2024-04-01T00:15:50.801 INFO:teuthology.orchestra.run.smithi064.stdout:checking for ld used by gcc... /bin/ld 2024-04-01T00:15:50.803 INFO:teuthology.orchestra.run.smithi064.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-01T00:15:50.808 INFO:teuthology.orchestra.run.smithi064.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-01T00:15:51.763 INFO:teuthology.orchestra.run.smithi064.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-01T00:15:51.763 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether ln -s works... yes 2024-04-01T00:15:51.774 INFO:teuthology.orchestra.run.smithi064.stdout:checking the maximum length of command line arguments... 1572864 2024-04-01T00:15:51.778 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-01T00:15:51.778 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking for /bin/ld option to reload object files... -r 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking for objdump... objdump 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to recognize dependent libraries... pass_all 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dlltool... no 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-01T00:15:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:checking for ar... ar 2024-04-01T00:15:51.820 INFO:teuthology.orchestra.run.smithi064.stdout:checking for archiver @FILE support... @ 2024-04-01T00:15:51.820 INFO:teuthology.orchestra.run.smithi064.stdout:checking for strip... strip 2024-04-01T00:15:51.820 INFO:teuthology.orchestra.run.smithi064.stdout:checking for ranlib... ranlib 2024-04-01T00:15:51.820 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gawk... gawk 2024-04-01T00:15:51.884 INFO:teuthology.orchestra.run.smithi064.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-01T00:15:51.887 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sysroot... no 2024-04-01T00:15:51.888 INFO:teuthology.orchestra.run.smithi064.stdout:checking for a working dd... /bin/dd 2024-04-01T00:15:51.890 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-01T00:15:51.903 INFO:teuthology.orchestra.run.smithi064.stdout:checking for mt... no 2024-04-01T00:15:51.905 INFO:teuthology.orchestra.run.smithi064.stdout:checking if : is a manifest tool... no 2024-04-01T00:15:51.946 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to run the C preprocessor... gcc -E 2024-04-01T00:15:52.124 INFO:teuthology.orchestra.run.smithi064.stdout:checking for ANSI C header files... yes 2024-04-01T00:15:52.144 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/types.h... yes 2024-04-01T00:15:52.171 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/stat.h... yes 2024-04-01T00:15:52.194 INFO:teuthology.orchestra.run.smithi064.stdout:checking for stdlib.h... yes 2024-04-01T00:15:52.218 INFO:teuthology.orchestra.run.smithi064.stdout:checking for string.h... yes 2024-04-01T00:15:52.249 INFO:teuthology.orchestra.run.smithi064.stdout:checking for memory.h... yes 2024-04-01T00:15:52.273 INFO:teuthology.orchestra.run.smithi064.stdout:checking for strings.h... yes 2024-04-01T00:15:52.308 INFO:teuthology.orchestra.run.smithi064.stdout:checking for inttypes.h... yes 2024-04-01T00:15:52.331 INFO:teuthology.orchestra.run.smithi064.stdout:checking for stdint.h... yes 2024-04-01T00:15:52.369 INFO:teuthology.orchestra.run.smithi064.stdout:checking for unistd.h... yes 2024-04-01T00:15:52.402 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dlfcn.h... yes 2024-04-01T00:15:52.405 INFO:teuthology.orchestra.run.smithi064.stdout:checking for objdir... .libs 2024-04-01T00:15:52.466 INFO:teuthology.orchestra.run.smithi064.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-01T00:15:52.466 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-01T00:15:52.480 INFO:teuthology.orchestra.run.smithi064.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-01T00:15:52.497 INFO:teuthology.orchestra.run.smithi064.stdout:checking if gcc static flag -static works... no 2024-04-01T00:15:52.517 INFO:teuthology.orchestra.run.smithi064.stdout:checking if gcc supports -c -o file.o... yes 2024-04-01T00:15:52.517 INFO:teuthology.orchestra.run.smithi064.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-01T00:15:52.523 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-01T00:15:52.547 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether -lc should be explicitly linked in... no 2024-04-01T00:15:52.603 INFO:teuthology.orchestra.run.smithi064.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-01T00:15:52.603 INFO:teuthology.orchestra.run.smithi064.stdout:checking how to hardcode library paths into programs... immediate 2024-04-01T00:15:52.604 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether stripping libraries is possible... yes 2024-04-01T00:15:52.605 INFO:teuthology.orchestra.run.smithi064.stdout:checking if libtool supports shared libraries... yes 2024-04-01T00:15:52.605 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether to build shared libraries... yes 2024-04-01T00:15:52.605 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether to build static libraries... yes 2024-04-01T00:15:52.610 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gcc... (cached) gcc 2024-04-01T00:15:52.625 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-01T00:15:52.625 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-01T00:15:52.626 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-01T00:15:52.626 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gmake... /usr/bin/gmake 2024-04-01T00:15:52.626 INFO:teuthology.orchestra.run.smithi064.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-01T00:15:52.627 INFO:teuthology.orchestra.run.smithi064.stdout:checking for tar... /bin/tar 2024-04-01T00:15:52.627 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gzip... /bin/gzip 2024-04-01T00:15:52.634 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether gcc -MM is supported... yes 2024-04-01T00:15:52.634 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sort... /bin/sort 2024-04-01T00:15:52.635 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether ln -s works... yes 2024-04-01T00:15:52.635 INFO:teuthology.orchestra.run.smithi064.stdout:checking for rpm... /bin/rpm 2024-04-01T00:15:52.638 INFO:teuthology.orchestra.run.smithi064.stdout:checking for rpmbuild... no 2024-04-01T00:15:52.662 INFO:teuthology.orchestra.run.smithi064.stdout:checking assert.h usability... yes 2024-04-01T00:15:52.670 INFO:teuthology.orchestra.run.smithi064.stdout:checking assert.h presence... yes 2024-04-01T00:15:52.670 INFO:teuthology.orchestra.run.smithi064.stdout:checking for assert.h... yes 2024-04-01T00:15:52.695 INFO:teuthology.orchestra.run.smithi064.stdout:checking bstring.h usability... no 2024-04-01T00:15:52.706 INFO:teuthology.orchestra.run.smithi064.stdout:checking bstring.h presence... no 2024-04-01T00:15:52.706 INFO:teuthology.orchestra.run.smithi064.stdout:checking for bstring.h... no 2024-04-01T00:15:52.730 INFO:teuthology.orchestra.run.smithi064.stdout:checking libgen.h usability... yes 2024-04-01T00:15:52.738 INFO:teuthology.orchestra.run.smithi064.stdout:checking libgen.h presence... yes 2024-04-01T00:15:52.739 INFO:teuthology.orchestra.run.smithi064.stdout:checking for libgen.h... yes 2024-04-01T00:15:52.775 INFO:teuthology.orchestra.run.smithi064.stdout:checking dirent.h usability... yes 2024-04-01T00:15:52.784 INFO:teuthology.orchestra.run.smithi064.stdout:checking dirent.h presence... yes 2024-04-01T00:15:52.784 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dirent.h... yes 2024-04-01T00:15:52.822 INFO:teuthology.orchestra.run.smithi064.stdout:checking errno.h usability... yes 2024-04-01T00:15:52.831 INFO:teuthology.orchestra.run.smithi064.stdout:checking errno.h presence... yes 2024-04-01T00:15:52.831 INFO:teuthology.orchestra.run.smithi064.stdout:checking for errno.h... yes 2024-04-01T00:15:52.866 INFO:teuthology.orchestra.run.smithi064.stdout:checking malloc.h usability... yes 2024-04-01T00:15:52.876 INFO:teuthology.orchestra.run.smithi064.stdout:checking malloc.h presence... yes 2024-04-01T00:15:52.877 INFO:teuthology.orchestra.run.smithi064.stdout:checking for malloc.h... yes 2024-04-01T00:15:52.903 INFO:teuthology.orchestra.run.smithi064.stdout:checking uuid.h usability... no 2024-04-01T00:15:52.913 INFO:teuthology.orchestra.run.smithi064.stdout:checking uuid.h presence... no 2024-04-01T00:15:52.913 INFO:teuthology.orchestra.run.smithi064.stdout:checking for uuid.h... no 2024-04-01T00:15:52.950 INFO:teuthology.orchestra.run.smithi064.stdout:checking uuid/uuid.h usability... yes 2024-04-01T00:15:52.959 INFO:teuthology.orchestra.run.smithi064.stdout:checking uuid/uuid.h presence... yes 2024-04-01T00:15:52.959 INFO:teuthology.orchestra.run.smithi064.stdout:checking for uuid/uuid.h... yes 2024-04-01T00:15:52.985 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/uuid.h usability... no 2024-04-01T00:15:52.995 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/uuid.h presence... no 2024-04-01T00:15:52.995 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/uuid.h... no 2024-04-01T00:15:53.028 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/file.h usability... yes 2024-04-01T00:15:53.036 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/file.h presence... yes 2024-04-01T00:15:53.036 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/file.h... yes 2024-04-01T00:15:53.071 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fcntl.h usability... yes 2024-04-01T00:15:53.080 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fcntl.h presence... yes 2024-04-01T00:15:53.081 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/fcntl.h... yes 2024-04-01T00:15:53.106 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/syssgi.h usability... no 2024-04-01T00:15:53.116 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/syssgi.h presence... no 2024-04-01T00:15:53.116 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/syssgi.h... no 2024-04-01T00:15:53.156 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/param.h usability... yes 2024-04-01T00:15:53.167 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/param.h presence... yes 2024-04-01T00:15:53.168 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/param.h... yes 2024-04-01T00:15:53.172 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/stat.h... (cached) yes 2024-04-01T00:15:53.198 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/statvfs.h usability... yes 2024-04-01T00:15:53.206 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/statvfs.h presence... yes 2024-04-01T00:15:53.207 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/statvfs.h... yes 2024-04-01T00:15:53.233 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/time.h usability... yes 2024-04-01T00:15:53.241 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/time.h presence... yes 2024-04-01T00:15:53.242 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/time.h... yes 2024-04-01T00:15:53.281 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/ioctl.h usability... yes 2024-04-01T00:15:53.289 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/ioctl.h presence... yes 2024-04-01T00:15:53.289 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/ioctl.h... yes 2024-04-01T00:15:53.320 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/wait.h usability... yes 2024-04-01T00:15:53.330 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/wait.h presence... yes 2024-04-01T00:15:53.330 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/wait.h... yes 2024-04-01T00:15:53.335 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/types.h... (cached) yes 2024-04-01T00:15:53.339 INFO:teuthology.orchestra.run.smithi064.stdout:checking for strings.h... (cached) yes 2024-04-01T00:15:53.366 INFO:teuthology.orchestra.run.smithi064.stdout:checking err.h usability... yes 2024-04-01T00:15:53.375 INFO:teuthology.orchestra.run.smithi064.stdout:checking err.h presence... yes 2024-04-01T00:15:53.375 INFO:teuthology.orchestra.run.smithi064.stdout:checking for err.h... yes 2024-04-01T00:15:53.402 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/falloc.h usability... yes 2024-04-01T00:15:53.409 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/falloc.h presence... yes 2024-04-01T00:15:53.409 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/falloc.h... yes 2024-04-01T00:15:53.436 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-01T00:15:53.447 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-01T00:15:53.447 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-01T00:15:53.471 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-01T00:15:53.481 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-01T00:15:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-01T00:15:53.504 INFO:teuthology.orchestra.run.smithi064.stdout:checking xfs/platform_defs.h usability... no 2024-04-01T00:15:53.514 INFO:teuthology.orchestra.run.smithi064.stdout:checking xfs/platform_defs.h presence... no 2024-04-01T00:15:53.515 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/platform_defs.h... no 2024-04-01T00:15:53.538 INFO:teuthology.orchestra.run.smithi064.stdout:checking btrfs/ioctl.h usability... no 2024-04-01T00:15:53.548 INFO:teuthology.orchestra.run.smithi064.stdout:checking btrfs/ioctl.h presence... no 2024-04-01T00:15:53.548 INFO:teuthology.orchestra.run.smithi064.stdout:checking for btrfs/ioctl.h... no 2024-04-01T00:15:53.572 INFO:teuthology.orchestra.run.smithi064.stdout:checking cifs/ioctl.h usability... no 2024-04-01T00:15:53.582 INFO:teuthology.orchestra.run.smithi064.stdout:checking cifs/ioctl.h presence... no 2024-04-01T00:15:53.582 INFO:teuthology.orchestra.run.smithi064.stdout:checking for cifs/ioctl.h... no 2024-04-01T00:15:53.608 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/mman.h usability... yes 2024-04-01T00:15:53.617 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/mman.h presence... yes 2024-04-01T00:15:53.617 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/mman.h... yes 2024-04-01T00:15:53.643 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/ext4.h usability... no 2024-04-01T00:15:53.653 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/ext4.h presence... no 2024-04-01T00:15:53.653 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/ext4.h... no 2024-04-01T00:15:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/xfs_log_format.h... no 2024-04-01T00:15:53.698 INFO:teuthology.orchestra.run.smithi064.stdout:checking for uuid_compare... no 2024-04-01T00:15:53.728 INFO:teuthology.orchestra.run.smithi064.stdout:checking for uuid_compare in -luuid... yes 2024-04-01T00:15:53.821 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/xfs.h... yes 2024-04-01T00:15:53.833 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/libxfs.h... no 2024-04-01T00:15:53.848 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-01T00:15:53.877 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/xqm.h... yes 2024-04-01T00:15:53.925 INFO:teuthology.orchestra.run.smithi064.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-01T00:15:53.948 INFO:teuthology.orchestra.run.smithi064.stdout:checking xfs/handle.h usability... yes 2024-04-01T00:15:53.955 INFO:teuthology.orchestra.run.smithi064.stdout:checking xfs/handle.h presence... yes 2024-04-01T00:15:53.955 INFO:teuthology.orchestra.run.smithi064.stdout:checking for xfs/handle.h... yes 2024-04-01T00:15:53.986 INFO:teuthology.orchestra.run.smithi064.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-01T00:15:54.009 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/acl.h usability... yes 2024-04-01T00:15:54.018 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/acl.h presence... yes 2024-04-01T00:15:54.018 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/acl.h... yes 2024-04-01T00:15:54.041 INFO:teuthology.orchestra.run.smithi064.stdout:checking acl/libacl.h usability... yes 2024-04-01T00:15:54.051 INFO:teuthology.orchestra.run.smithi064.stdout:checking acl/libacl.h presence... yes 2024-04-01T00:15:54.051 INFO:teuthology.orchestra.run.smithi064.stdout:checking for acl/libacl.h... yes 2024-04-01T00:15:54.081 INFO:teuthology.orchestra.run.smithi064.stdout:checking for acl_init in -lacl... yes 2024-04-01T00:15:54.103 INFO:teuthology.orchestra.run.smithi064.stdout:checking gdbm-ndbm.h usability... no 2024-04-01T00:15:54.114 INFO:teuthology.orchestra.run.smithi064.stdout:checking gdbm-ndbm.h presence... no 2024-04-01T00:15:54.114 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gdbm-ndbm.h... no 2024-04-01T00:15:54.136 INFO:teuthology.orchestra.run.smithi064.stdout:checking gdbm/ndbm.h usability... yes 2024-04-01T00:15:54.146 INFO:teuthology.orchestra.run.smithi064.stdout:checking gdbm/ndbm.h presence... yes 2024-04-01T00:15:54.146 INFO:teuthology.orchestra.run.smithi064.stdout:checking for gdbm/ndbm.h... yes 2024-04-01T00:15:54.179 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_open in -lgdbm... no 2024-04-01T00:15:54.213 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-01T00:15:54.246 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_store in -lgdbm... no 2024-04-01T00:15:54.280 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_close in -lgdbm... no 2024-04-01T00:15:54.330 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-01T00:15:54.361 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-01T00:15:54.392 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-01T00:15:54.424 INFO:teuthology.orchestra.run.smithi064.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-01T00:15:54.447 INFO:teuthology.orchestra.run.smithi064.stdout:checking libaio.h usability... yes 2024-04-01T00:15:54.457 INFO:teuthology.orchestra.run.smithi064.stdout:checking libaio.h presence... yes 2024-04-01T00:15:54.457 INFO:teuthology.orchestra.run.smithi064.stdout:checking for libaio.h... yes 2024-04-01T00:15:54.479 INFO:teuthology.orchestra.run.smithi064.stdout:checking liburing.h usability... no 2024-04-01T00:15:54.489 INFO:teuthology.orchestra.run.smithi064.stdout:checking liburing.h presence... no 2024-04-01T00:15:54.490 INFO:teuthology.orchestra.run.smithi064.stdout:checking for liburing.h... no 2024-04-01T00:15:54.512 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/capability.h usability... yes 2024-04-01T00:15:54.522 INFO:teuthology.orchestra.run.smithi064.stdout:checking sys/capability.h presence... yes 2024-04-01T00:15:54.522 INFO:teuthology.orchestra.run.smithi064.stdout:checking for sys/capability.h... yes 2024-04-01T00:15:54.546 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/fiemap.h usability... yes 2024-04-01T00:15:54.553 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/fiemap.h presence... yes 2024-04-01T00:15:54.553 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/fiemap.h... yes 2024-04-01T00:15:54.587 INFO:teuthology.orchestra.run.smithi064.stdout:checking for fallocate... yes 2024-04-01T00:15:54.620 INFO:teuthology.orchestra.run.smithi064.stdout:checking for open_by_handle_at... yes 2024-04-01T00:15:54.643 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/fs.h usability... yes 2024-04-01T00:15:54.651 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/fs.h presence... yes 2024-04-01T00:15:54.651 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/fs.h... yes 2024-04-01T00:15:54.673 INFO:teuthology.orchestra.run.smithi064.stdout:checking btrfsutil.h usability... no 2024-04-01T00:15:54.683 INFO:teuthology.orchestra.run.smithi064.stdout:checking btrfsutil.h presence... no 2024-04-01T00:15:54.683 INFO:teuthology.orchestra.run.smithi064.stdout:checking for btrfsutil.h... no 2024-04-01T00:15:54.718 INFO:teuthology.orchestra.run.smithi064.stdout:checking for copy_file_range... yes 2024-04-01T00:15:54.754 INFO:teuthology.orchestra.run.smithi064.stdout:checking for SEEK_DATA... yes 2024-04-01T00:15:54.802 INFO:teuthology.orchestra.run.smithi064.stdout:checking for BMV_OF_SHARED... yes 2024-04-01T00:15:54.837 INFO:teuthology.orchestra.run.smithi064.stdout:checking for nftw... yes 2024-04-01T00:15:54.876 INFO:teuthology.orchestra.run.smithi064.stdout:checking for RLIMIT_NOFILE... yes 2024-04-01T00:15:54.909 INFO:teuthology.orchestra.run.smithi064.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-01T00:15:54.942 INFO:teuthology.orchestra.run.smithi064.stdout:checking for FICLONE... yes 2024-04-01T00:15:54.974 INFO:teuthology.orchestra.run.smithi064.stdout:checking for renameat2... yes 2024-04-01T00:15:55.007 INFO:teuthology.orchestra.run.smithi064.stdout:checking for reallocarray... yes 2024-04-01T00:15:55.036 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct mount_attr... yes 2024-04-01T00:15:55.070 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-01T00:15:55.099 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-01T00:15:55.130 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-01T00:15:55.161 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-01T00:15:55.192 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-01T00:15:55.223 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-01T00:15:55.254 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-01T00:15:55.280 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/btrfs.h usability... yes 2024-04-01T00:15:55.288 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/btrfs.h presence... yes 2024-04-01T00:15:55.288 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/btrfs.h... yes 2024-04-01T00:15:55.324 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-01T00:15:55.333 INFO:teuthology.orchestra.run.smithi064.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-01T00:15:55.333 INFO:teuthology.orchestra.run.smithi064.stdout:checking for linux/btrfs_tree.h... yes 2024-04-01T00:15:55.353 INFO:teuthology.orchestra.run.smithi064.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-01T00:15:55.374 INFO:teuthology.orchestra.run.smithi064.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-01T00:15:55.390 INFO:teuthology.orchestra.run.smithi064.stdout:configure: creating ./config.status 2024-04-01T00:15:55.675 INFO:teuthology.orchestra.run.smithi064.stdout:config.status: creating include/builddefs 2024-04-01T00:15:55.684 INFO:teuthology.orchestra.run.smithi064.stdout:config.status: creating include/config.h 2024-04-01T00:15:55.692 INFO:teuthology.orchestra.run.smithi064.stdout:config.status: executing libtool commands 2024-04-01T00:15:55.729 INFO:teuthology.orchestra.run.smithi064.stdout:Building include 2024-04-01T00:15:55.732 INFO:teuthology.orchestra.run.smithi064.stdout:Building lib 2024-04-01T00:15:55.735 INFO:teuthology.orchestra.run.smithi064.stdout: [LTDEP] 2024-04-01T00:15:55.875 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dataascii.lo 2024-04-01T00:15:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] databin.lo 2024-04-01T00:15:56.116 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] datapid.lo 2024-04-01T00:15:56.209 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] file_lock.lo 2024-04-01T00:15:56.342 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] forker.lo 2024-04-01T00:15:56.462 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] pattern.lo 2024-04-01T00:15:56.571 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] open_flags.lo 2024-04-01T00:15:56.697 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] random_range.lo 2024-04-01T00:15:56.907 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] string_to_tokens.lo 2024-04-01T00:15:56.999 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] str_to_bytes.lo 2024-04-01T00:15:57.117 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] tlibio.lo 2024-04-01T00:15:57.337 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] write_log.lo 2024-04-01T00:15:57.555 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] random.lo 2024-04-01T00:15:57.651 INFO:teuthology.orchestra.run.smithi064.stdout: [LD] 2024-04-01T00:15:57.728 INFO:teuthology.orchestra.run.smithi064.stdout:Building ltp 2024-04-01T00:15:57.730 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:15:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] doio 2024-04-01T00:15:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fsstress 2024-04-01T00:15:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fsx 2024-04-01T00:16:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] growfiles 2024-04-01T00:16:01.008 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] iogen 2024-04-01T00:16:01.353 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-stress 2024-04-01T00:16:01.741 INFO:teuthology.orchestra.run.smithi064.stdout:Building src 2024-04-01T00:16:01.746 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:16:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:16:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:16:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:16:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: [DEP] 2024-04-01T00:16:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dirstress 2024-04-01T00:16:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fill 2024-04-01T00:16:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fill2 2024-04-01T00:16:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] getpagesize 2024-04-01T00:16:04.339 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] holes 2024-04-01T00:16:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] lstat64 2024-04-01T00:16:04.665 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] nametest 2024-04-01T00:16:04.875 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] permname 2024-04-01T00:16:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] randholes 2024-04-01T00:16:05.269 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] runas 2024-04-01T00:16:05.417 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] truncfile 2024-04-01T00:16:05.570 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] usemem 2024-04-01T00:16:05.697 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] mmapcat 2024-04-01T00:16:05.820 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] append_reader 2024-04-01T00:16:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] append_writer 2024-04-01T00:16:06.072 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dirperf 2024-04-01T00:16:06.269 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] metaperf 2024-04-01T00:16:06.511 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] devzero 2024-04-01T00:16:06.659 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] feature 2024-04-01T00:16:06.850 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] alloc 2024-04-01T00:16:07.028 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fault 2024-04-01T00:16:07.165 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fstest 2024-04-01T00:16:07.371 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_access_root 2024-04-01T00:16:07.500 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] godown 2024-04-01T00:16:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] resvtest 2024-04-01T00:16:07.800 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] writemod 2024-04-01T00:16:07.921 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] writev_on_pagefault 2024-04-01T00:16:08.054 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] makeextents 2024-04-01T00:16:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] itrash 2024-04-01T00:16:08.333 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] rename 2024-04-01T00:16:08.449 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] multi_open_unlink 2024-04-01T00:16:08.587 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] unwritten_sync 2024-04-01T00:16:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] genhashnames 2024-04-01T00:16:08.907 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_holes 2024-04-01T00:16:09.031 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_writev 2024-04-01T00:16:09.155 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_truncate_cmtime 2024-04-01T00:16:09.288 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dirhash_collide 2024-04-01T00:16:09.444 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_rename_overwrite 2024-04-01T00:16:09.562 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] holetest 2024-04-01T00:16:09.760 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_truncate_self 2024-04-01T00:16:09.878 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] af_unix 2024-04-01T00:16:10.005 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_stale_pmd 2024-04-01T00:16:10.133 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_cow_race 2024-04-01T00:16:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_fallocate 2024-04-01T00:16:10.398 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fsync-err 2024-04-01T00:16:10.558 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_write_ro 2024-04-01T00:16:10.685 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-01T00:16:10.819 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-01T00:16:10.952 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_ofd_locks 2024-04-01T00:16:11.120 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_collision 2024-04-01T00:16:11.291 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] mmap-write-concurrent 2024-04-01T00:16:11.435 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_get_file_time 2024-04-01T00:16:11.558 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_create_short_dirs 2024-04-01T00:16:11.701 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_create_long_dirs 2024-04-01T00:16:11.842 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_enospc 2024-04-01T00:16:12.002 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_writev_overlap 2024-04-01T00:16:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] checkpoint_journal 2024-04-01T00:16:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] mmap-rw-fault 2024-04-01T00:16:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] allocstale 2024-04-01T00:16:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_cow_memory_failure 2024-04-01T00:16:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fake-dump-rootino 2024-04-01T00:16:12.845 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dio-buf-fault 2024-04-01T00:16:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] rewinddir-test 2024-04-01T00:16:13.117 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] readdir-while-renames 2024-04-01T00:16:13.257 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] xfsctl 2024-04-01T00:16:13.417 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] bstat 2024-04-01T00:16:13.597 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mtab 2024-04-01T00:16:13.759 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] getdevicesize 2024-04-01T00:16:13.878 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] preallo_rw_pattern_reader 2024-04-01T00:16:14.012 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] preallo_rw_pattern_writer 2024-04-01T00:16:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] ftrunc 2024-04-01T00:16:14.268 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] trunc 2024-04-01T00:16:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fs_perms 2024-04-01T00:16:14.543 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] testx 2024-04-01T00:16:14.653 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] looptest 2024-04-01T00:16:14.819 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] locktest 2024-04-01T00:16:15.099 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] unwritten_mmap 2024-04-01T00:16:15.237 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] bulkstat_unlink_test 2024-04-01T00:16:15.409 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] deduperace 2024-04-01T00:16:15.577 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] bulkstat_unlink_test_modified 2024-04-01T00:16:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_dir_offset 2024-04-01T00:16:15.872 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_futimens 2024-04-01T00:16:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_immutable 2024-04-01T00:16:16.564 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] stale_handle 2024-04-01T00:16:16.708 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] pwrite_mmap_blocked 2024-04-01T00:16:16.836 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_dir_offset2 2024-04-01T00:16:16.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:16:16.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:16:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] seek_sanity_test 2024-04-01T00:16:17.417 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] seek_copy_test 2024-04-01T00:16:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_readdir_1 2024-04-01T00:16:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_readdir_2 2024-04-01T00:16:17.856 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fsync-tester 2024-04-01T00:16:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] nsexec 2024-04-01T00:16:18.230 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] cloner 2024-04-01T00:16:18.382 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] renameat2 2024-04-01T00:16:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_getcwd 2024-04-01T00:16:18.656 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] e4compact 2024-04-01T00:16:18.833 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] test-nextquota 2024-04-01T00:16:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] punch-alternating 2024-04-01T00:16:19.124 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-01T00:16:19.266 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] listxattr 2024-04-01T00:16:19.390 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dio-interleaved 2024-04-01T00:16:19.526 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_dir_type 2024-04-01T00:16:19.658 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dio-invalidate-cache 2024-04-01T00:16:19.834 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] stat_test 2024-04-01T00:16:20.099 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_encrypted_d_revalidate 2024-04-01T00:16:20.238 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] attr_replace_test 2024-04-01T00:16:20.377 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] swapon 2024-04-01T00:16:20.517 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] mkswap 2024-04-01T00:16:20.644 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_attr_corruption 2024-04-01T00:16:20.771 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_open_tmpfiles 2024-04-01T00:16:20.925 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fscrypt-crypt-util 2024-04-01T00:16:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] bulkstat_null_ocount 2024-04-01T00:16:21.671 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] splice-test 2024-04-01T00:16:21.827 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] chprojid_fail 2024-04-01T00:16:21.964 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] detached_mounts_propagation 2024-04-01T00:16:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] ext4_resize 2024-04-01T00:16:22.249 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_readdir_3 2024-04-01T00:16:22.417 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] splice2pipe 2024-04-01T00:16:22.555 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] uuid_ioctl 2024-04-01T00:16:22.687 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-01T00:16:22.826 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fiemap-tester 2024-04-01T00:16:23.065 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_stripealign 2024-04-01T00:16:23.195 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] open_by_handle 2024-04-01T00:16:23.416 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] dbtest 2024-04-01T00:16:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_mmap_dio 2024-04-01T00:16:23.772 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] xfsfind 2024-04-01T00:16:23.941 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] t_reflink_read_race 2024-04-01T00:16:24.106 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] fssum 2024-04-01T00:16:24.510 INFO:teuthology.orchestra.run.smithi064.stdout:Building log-writes 2024-04-01T00:16:24.515 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] replay-log 2024-04-01T00:16:24.799 INFO:teuthology.orchestra.run.smithi064.stdout:Building perf 2024-04-01T00:16:24.802 INFO:teuthology.orchestra.run.smithi064.stdout:Building aio-dio-regress 2024-04-01T00:16:24.808 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aiocp 2024-04-01T00:16:25.009 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-01T00:16:25.152 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-append-write-read-race 2024-04-01T00:16:25.301 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-cow-race 2024-04-01T00:16:25.430 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-cycle-write 2024-04-01T00:16:25.589 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-eof-race 2024-04-01T00:16:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-extend-stat 2024-04-01T00:16:25.886 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-fcntl-race 2024-04-01T00:16:26.025 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-hole-filling-race 2024-04-01T00:16:26.149 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-invalidate-failure 2024-04-01T00:16:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-invalidate-readahead 2024-04-01T00:16:26.427 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aiodio_sparse2 2024-04-01T00:16:26.620 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-subblock-eof-read 2024-04-01T00:16:26.740 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-dio-write-verify 2024-04-01T00:16:26.911 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-01T00:16:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-01T00:16:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] aio-last-ref-held-by-io 2024-04-01T00:16:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:Building vfs 2024-04-01T00:16:27.286 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] vfstest 2024-04-01T00:16:30.725 INFO:teuthology.orchestra.run.smithi064.stdout: [CC] mount-idmapped 2024-04-01T00:16:31.070 INFO:teuthology.orchestra.run.smithi064.stdout:Building m4 2024-04-01T00:16:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:Building common 2024-04-01T00:16:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:Building tests 2024-04-01T00:16:31.081 INFO:teuthology.orchestra.run.smithi064.stdout:Building btrfs 2024-04-01T00:16:31.087 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/btrfs/group.list 2024-04-01T00:16:31.145 INFO:teuthology.orchestra.run.smithi064.stdout:Building ceph 2024-04-01T00:16:31.150 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ceph/group.list 2024-04-01T00:16:31.163 INFO:teuthology.orchestra.run.smithi064.stdout:Building cifs 2024-04-01T00:16:31.168 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/cifs/group.list 2024-04-01T00:16:31.180 INFO:teuthology.orchestra.run.smithi064.stdout:Building ext4 2024-04-01T00:16:31.185 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ext4/group.list 2024-04-01T00:16:31.221 INFO:teuthology.orchestra.run.smithi064.stdout:Building f2fs 2024-04-01T00:16:31.225 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/f2fs/group.list 2024-04-01T00:16:31.242 INFO:teuthology.orchestra.run.smithi064.stdout:Building generic 2024-04-01T00:16:31.248 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/generic/group.list 2024-04-01T00:16:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:Building nfs 2024-04-01T00:16:31.345 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/nfs/group.list 2024-04-01T00:16:31.358 INFO:teuthology.orchestra.run.smithi064.stdout:Building ocfs2 2024-04-01T00:16:31.363 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ocfs2/group.list 2024-04-01T00:16:31.376 INFO:teuthology.orchestra.run.smithi064.stdout:Building overlay 2024-04-01T00:16:31.381 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/overlay/group.list 2024-04-01T00:16:31.419 INFO:teuthology.orchestra.run.smithi064.stdout:Building perf 2024-04-01T00:16:31.424 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/perf/group.list 2024-04-01T00:16:31.435 INFO:teuthology.orchestra.run.smithi064.stdout:Building selftest 2024-04-01T00:16:31.439 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/selftest/group.list 2024-04-01T00:16:31.454 INFO:teuthology.orchestra.run.smithi064.stdout:Building shared 2024-04-01T00:16:31.456 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/shared/group.list 2024-04-01T00:16:31.472 INFO:teuthology.orchestra.run.smithi064.stdout:Building tmpfs 2024-04-01T00:16:31.477 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/tmpfs/group.list 2024-04-01T00:16:31.489 INFO:teuthology.orchestra.run.smithi064.stdout:Building udf 2024-04-01T00:16:31.494 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/udf/group.list 2024-04-01T00:16:31.505 INFO:teuthology.orchestra.run.smithi064.stdout:Building xfs 2024-04-01T00:16:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/xfs/group.list 2024-04-01T00:16:31.606 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo make install) 2024-04-01T00:16:31.643 INFO:teuthology.orchestra.run.smithi064.stdout:Building include 2024-04-01T00:16:31.646 INFO:teuthology.orchestra.run.smithi064.stdout:Building lib 2024-04-01T00:16:31.650 INFO:teuthology.orchestra.run.smithi064.stdout:Building ltp 2024-04-01T00:16:31.654 INFO:teuthology.orchestra.run.smithi064.stdout:Building src 2024-04-01T00:16:31.687 INFO:teuthology.orchestra.run.smithi064.stdout:Building log-writes 2024-04-01T00:16:31.690 INFO:teuthology.orchestra.run.smithi064.stdout:Building perf 2024-04-01T00:16:31.693 INFO:teuthology.orchestra.run.smithi064.stdout:Building aio-dio-regress 2024-04-01T00:16:31.697 INFO:teuthology.orchestra.run.smithi064.stdout:Building vfs 2024-04-01T00:16:31.701 INFO:teuthology.orchestra.run.smithi064.stdout:Building m4 2024-04-01T00:16:31.704 INFO:teuthology.orchestra.run.smithi064.stdout:Building common 2024-04-01T00:16:31.707 INFO:teuthology.orchestra.run.smithi064.stdout:Building tests 2024-04-01T00:16:31.712 INFO:teuthology.orchestra.run.smithi064.stdout:Building btrfs 2024-04-01T00:16:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/btrfs/group.list 2024-04-01T00:16:31.775 INFO:teuthology.orchestra.run.smithi064.stdout:Building ceph 2024-04-01T00:16:31.780 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ceph/group.list 2024-04-01T00:16:31.793 INFO:teuthology.orchestra.run.smithi064.stdout:Building cifs 2024-04-01T00:16:31.798 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/cifs/group.list 2024-04-01T00:16:31.810 INFO:teuthology.orchestra.run.smithi064.stdout:Building ext4 2024-04-01T00:16:31.815 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ext4/group.list 2024-04-01T00:16:31.851 INFO:teuthology.orchestra.run.smithi064.stdout:Building f2fs 2024-04-01T00:16:31.855 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/f2fs/group.list 2024-04-01T00:16:31.872 INFO:teuthology.orchestra.run.smithi064.stdout:Building generic 2024-04-01T00:16:31.878 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/generic/group.list 2024-04-01T00:16:31.969 INFO:teuthology.orchestra.run.smithi064.stdout:Building nfs 2024-04-01T00:16:31.973 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/nfs/group.list 2024-04-01T00:16:31.987 INFO:teuthology.orchestra.run.smithi064.stdout:Building ocfs2 2024-04-01T00:16:31.992 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ocfs2/group.list 2024-04-01T00:16:32.004 INFO:teuthology.orchestra.run.smithi064.stdout:Building overlay 2024-04-01T00:16:32.009 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/overlay/group.list 2024-04-01T00:16:32.048 INFO:teuthology.orchestra.run.smithi064.stdout:Building perf 2024-04-01T00:16:32.052 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/perf/group.list 2024-04-01T00:16:32.063 INFO:teuthology.orchestra.run.smithi064.stdout:Building selftest 2024-04-01T00:16:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/selftest/group.list 2024-04-01T00:16:32.080 INFO:teuthology.orchestra.run.smithi064.stdout:Building shared 2024-04-01T00:16:32.084 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/shared/group.list 2024-04-01T00:16:32.101 INFO:teuthology.orchestra.run.smithi064.stdout:Building tmpfs 2024-04-01T00:16:32.105 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/tmpfs/group.list 2024-04-01T00:16:32.118 INFO:teuthology.orchestra.run.smithi064.stdout:Building udf 2024-04-01T00:16:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/udf/group.list 2024-04-01T00:16:32.133 INFO:teuthology.orchestra.run.smithi064.stdout:Building xfs 2024-04-01T00:16:32.140 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/xfs/group.list 2024-04-01T00:16:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-01T00:16:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-01T00:16:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-01T00:16:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-01T00:16:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-01T00:16:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-01T00:16:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-01T00:16:32.309 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-01T00:16:32.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-01T00:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout:Building log-writes 2024-04-01T00:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout:Building perf 2024-04-01T00:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout:Building aio-dio-regress 2024-04-01T00:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout:Building vfs 2024-04-01T00:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake -C log-writes install 2024-04-01T00:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-01T00:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-01T00:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake -C perf install 2024-04-01T00:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-01T00:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-01T00:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-01T00:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-01T00:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-01T00:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake -C vfs install 2024-04-01T00:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-01T00:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-01T00:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-01T00:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-01T00:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-01T00:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-01T00:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-01T00:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-01T00:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-01T00:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-01T00:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-01T00:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-01T00:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/btrfs/ install 2024-04-01T00:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/btrfs/group.list 2024-04-01T00:16:33.880 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-01T00:16:33.888 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 /var/lib/xfstests/tests/btrfs 2024-04-01T00:16:34.644 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-01T00:16:34.652 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out /var/lib/xfstests/tests/btrfs 2024-04-01T00:16:35.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ceph/ install 2024-04-01T00:16:35.411 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ceph/group.list 2024-04-01T00:16:35.424 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-01T00:16:35.432 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-01T00:16:35.449 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-01T00:16:35.457 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-01T00:16:35.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/cifs/ install 2024-04-01T00:16:35.477 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/cifs/group.list 2024-04-01T00:16:35.488 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-01T00:16:35.496 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-01T00:16:35.504 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-01T00:16:35.512 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-01T00:16:35.520 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ext4/ install 2024-04-01T00:16:35.523 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ext4/group.list 2024-04-01T00:16:35.558 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-01T00:16:35.566 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-01T00:16:35.730 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-01T00:16:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-01T00:16:35.910 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/f2fs/ install 2024-04-01T00:16:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/f2fs/group.list 2024-04-01T00:16:35.928 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-01T00:16:35.936 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-01T00:16:35.946 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-01T00:16:35.954 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-01T00:16:35.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/generic/ install 2024-04-01T00:16:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/generic/group.list 2024-04-01T00:16:36.061 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-01T00:16:36.069 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-04-01T00:16:37.829 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-01T00:16:37.837 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-04-01T00:16:39.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/nfs/ install 2024-04-01T00:16:39.611 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/nfs/group.list 2024-04-01T00:16:39.623 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-01T00:16:39.631 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-01T00:16:39.639 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-01T00:16:39.647 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-01T00:16:39.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ocfs2/ install 2024-04-01T00:16:39.658 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/ocfs2/group.list 2024-04-01T00:16:39.669 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-01T00:16:39.677 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-01T00:16:39.685 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-01T00:16:39.693 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-01T00:16:39.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/overlay/ install 2024-04-01T00:16:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/overlay/group.list 2024-04-01T00:16:39.741 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-01T00:16:39.749 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-01T00:16:39.998 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-01T00:16:40.008 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-01T00:16:40.259 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/perf/ install 2024-04-01T00:16:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/perf/group.list 2024-04-01T00:16:40.273 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-01T00:16:40.279 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-01T00:16:40.287 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-01T00:16:40.295 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-01T00:16:40.303 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/selftest/ install 2024-04-01T00:16:40.306 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/selftest/group.list 2024-04-01T00:16:40.317 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-01T00:16:40.325 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-01T00:16:40.345 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-01T00:16:40.352 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-01T00:16:40.372 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/shared/ install 2024-04-01T00:16:40.374 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/shared/group.list 2024-04-01T00:16:40.390 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-01T00:16:40.398 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-01T00:16:40.410 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-01T00:16:40.419 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-01T00:16:40.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/tmpfs/ install 2024-04-01T00:16:40.433 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/tmpfs/group.list 2024-04-01T00:16:40.445 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-01T00:16:40.453 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-01T00:16:40.461 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-01T00:16:40.469 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-01T00:16:40.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/udf/ install 2024-04-01T00:16:40.480 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/udf/group.list 2024-04-01T00:16:40.490 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-01T00:16:40.498 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-01T00:16:40.506 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-01T00:16:40.513 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-01T00:16:40.522 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/xfs/ install 2024-04-01T00:16:40.525 INFO:teuthology.orchestra.run.smithi064.stdout: [GROUP] /tmp/tmp.ALJJcUYdNQxfstests-dev/tests/xfs/group.list 2024-04-01T00:16:40.618 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-01T00:16:40.626 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-01T00:16:42.284 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-01T00:16:42.293 INFO:teuthology.orchestra.run.smithi064.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-01T00:16:43.975 INFO:teuthology.orchestra.run.smithi064.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-01T00:16:43.982 INFO:teuthology.orchestra.run.smithi064.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-01T00:16:43.990 INFO:teuthology.orchestra.run.smithi064.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-01T00:16:43.998 INFO:teuthology.orchestra.run.smithi064.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-01T00:16:44.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T00:16:44.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec ls tests/generic) 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:001 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:001.out 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:002 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:002.out 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:003 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:003.out 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:004 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:004.out 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:005 2024-04-01T00:16:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:005.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:006 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:006.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:007 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:007.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:008 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:008.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:009 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:009.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:010 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:010.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:011 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:011.out 2024-04-01T00:16:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:012 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:012.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:013 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:013.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:014 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:014.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:015 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:015.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:016 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:016.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:017 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:017.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:018 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:018.out 2024-04-01T00:16:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:019 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:019.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:020 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:020.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:021 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:021.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:022 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:022.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:023 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:023.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:024 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:024.out 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:025 2024-04-01T00:16:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:025.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:026 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:026.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:027 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:027.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:028 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:028.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:029 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:029.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:030 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:030.out 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:031 2024-04-01T00:16:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:031.out 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:032 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:032.out 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:033 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:033.out 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:034 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:034.out 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:035 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:035.cfg 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:035.out.default 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:035.out.nfs 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:036 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:036.out 2024-04-01T00:16:44.070 INFO:teuthology.orchestra.run.smithi064.stdout:037 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:037.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:038 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:038.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:039 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:039.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:040 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:040.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:041 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:041.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:042 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:042.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:043 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:043.out 2024-04-01T00:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:044 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:044.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:045 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:045.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:046 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:046.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:047 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:047.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:048 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:048.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:049 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:049.out 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:050 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:050.cfg 2024-04-01T00:16:44.072 INFO:teuthology.orchestra.run.smithi064.stdout:050.out.default 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:050.out.nojournal 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:050.out.xfsquota 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:051 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:051.out 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:052 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:052.out 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:053 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:053.out 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:054 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:054.out 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:055 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:055.out 2024-04-01T00:16:44.073 INFO:teuthology.orchestra.run.smithi064.stdout:056 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:056.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:057 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:057.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:058 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:058.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:059 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:059.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:060 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:060.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:061 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:061.out 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:062 2024-04-01T00:16:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:062.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:063 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:063.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:064 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:064.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:065 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:065.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:066 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:066.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:067 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:067.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:068 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:068.out 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:069 2024-04-01T00:16:44.075 INFO:teuthology.orchestra.run.smithi064.stdout:069.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:070 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:070.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:071 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:071.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:072 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:072.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:073 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:073.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:074 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:074.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:075 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:075.out 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:076 2024-04-01T00:16:44.076 INFO:teuthology.orchestra.run.smithi064.stdout:076.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:077 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:077.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:078 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:078.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:079 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:079.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:080 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:080.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:081 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:081.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:082 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:082.out 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:083 2024-04-01T00:16:44.077 INFO:teuthology.orchestra.run.smithi064.stdout:083.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:084 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:084.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:085 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:085.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:086 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:086.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:087 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:087.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:088 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:088.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:089 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:089.out 2024-04-01T00:16:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:090 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:090.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:091 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:091.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:092 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:092.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:093 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:093.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:094 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:094.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:095 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:095.out 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:096 2024-04-01T00:16:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:096.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:097 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:097.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:098 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:098.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:099 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:099.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:100 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:100.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:101 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:101.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:102 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:102.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:103 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:103.out 2024-04-01T00:16:44.080 INFO:teuthology.orchestra.run.smithi064.stdout:104 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:104.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:105 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:105.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:106 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:106.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:107 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:107.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:108 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:108.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:109 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:109.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:110 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:110.out 2024-04-01T00:16:44.081 INFO:teuthology.orchestra.run.smithi064.stdout:111 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:111.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:112 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:112.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:113 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:113.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:114 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:114.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:115 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:115.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:116 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:116.out 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:117 2024-04-01T00:16:44.082 INFO:teuthology.orchestra.run.smithi064.stdout:117.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:118 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:118.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:119 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:119.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:120 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:120.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:121 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:121.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:122 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:122.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:123 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:123.out 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:124 2024-04-01T00:16:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:124.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:125 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:125.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:126 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:126.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:127 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:127.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:128 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:128.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:129 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:129.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:130 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:130.out 2024-04-01T00:16:44.084 INFO:teuthology.orchestra.run.smithi064.stdout:131 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:131.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:132 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:132.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:133 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:133.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:134 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:134.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:135 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:135.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:136 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:136.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:137 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:137.out 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:138 2024-04-01T00:16:44.085 INFO:teuthology.orchestra.run.smithi064.stdout:138.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:139 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:139.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:140 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:140.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:141 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:141.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:142 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:142.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:143 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:143.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:144 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:144.out 2024-04-01T00:16:44.086 INFO:teuthology.orchestra.run.smithi064.stdout:145 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:145.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:146 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:146.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:147 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:147.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:148 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:148.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:149 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:149.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:150 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:150.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:151 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:151.out 2024-04-01T00:16:44.087 INFO:teuthology.orchestra.run.smithi064.stdout:152 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:152.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:153 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:153.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:154 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:154.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:155 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:155.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:156 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:156.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:157 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:157.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:158 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:158.out 2024-04-01T00:16:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:159 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:159.out 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:160 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:160.out 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:161 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:161.out 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:162 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:162.out 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:163 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:163.out 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:164 2024-04-01T00:16:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:164.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:165 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:165.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:166 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:166.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:167 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:167.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:168 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:168.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:169 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:169.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:170 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:170.out 2024-04-01T00:16:44.090 INFO:teuthology.orchestra.run.smithi064.stdout:171 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:171.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:172 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:172.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:173 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:173.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:174 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:174.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:175 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:175.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:176 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:176.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:177 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:177.out 2024-04-01T00:16:44.091 INFO:teuthology.orchestra.run.smithi064.stdout:178 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:178.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:179 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:179.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:180 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:180.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:181 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:181.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:182 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:182.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:183 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:183.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:184 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:184.out 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:185 2024-04-01T00:16:44.092 INFO:teuthology.orchestra.run.smithi064.stdout:185.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:186 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:186.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:187 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:187.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:188 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:188.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:189 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:189.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:190 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:190.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:191 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:191.out 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:192 2024-04-01T00:16:44.093 INFO:teuthology.orchestra.run.smithi064.stdout:192.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:193 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:193.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:194 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:194.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:195 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:195.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:196 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:196.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:197 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:197.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:198 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:198.out 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:199 2024-04-01T00:16:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:199.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:200 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:200.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:201 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:201.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:202 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:202.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:203 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:203.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:204 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:204.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:205 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:205.out 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:206 2024-04-01T00:16:44.095 INFO:teuthology.orchestra.run.smithi064.stdout:206.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:207 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:207.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:208 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:208.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:209 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:209.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:210 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:210.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:211 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:211.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:212 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:212.out 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:213 2024-04-01T00:16:44.096 INFO:teuthology.orchestra.run.smithi064.stdout:213.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:214 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:214.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:215 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:215.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:216 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:216.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:217 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:217.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:218 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:218.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:219 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:219.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:220 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:220.out 2024-04-01T00:16:44.097 INFO:teuthology.orchestra.run.smithi064.stdout:221 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:221.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:222 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:222.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:223 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:223.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:224 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:224.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:225 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:225.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:226 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:226.out 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:227 2024-04-01T00:16:44.098 INFO:teuthology.orchestra.run.smithi064.stdout:227.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:228 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:228.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:229 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:229.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:230 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:230.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:231 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:231.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:232 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:232.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:233 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:233.out 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:234 2024-04-01T00:16:44.099 INFO:teuthology.orchestra.run.smithi064.stdout:234.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:235 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:235.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:236 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:236.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:237 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:237.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:238 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:238.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:239 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:239.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:240 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:240.out 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:241 2024-04-01T00:16:44.100 INFO:teuthology.orchestra.run.smithi064.stdout:241.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:242 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:242.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:243 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:243.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:244 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:244.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:245 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:245.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:246 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:246.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:247 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:247.out 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:248 2024-04-01T00:16:44.101 INFO:teuthology.orchestra.run.smithi064.stdout:248.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:249 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:249.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:250 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:250.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:251 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:251.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:252 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:252.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:253 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:253.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:254 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:254.out 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:255 2024-04-01T00:16:44.102 INFO:teuthology.orchestra.run.smithi064.stdout:255.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:256 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:256.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:257 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:257.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:258 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:258.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:259 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:259.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:260 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:260.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:261 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:261.out 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:262 2024-04-01T00:16:44.103 INFO:teuthology.orchestra.run.smithi064.stdout:262.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:263 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:263.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:264 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:264.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:265 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:265.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:266 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:266.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:267 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:267.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:268 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:268.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:269 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:269.out 2024-04-01T00:16:44.104 INFO:teuthology.orchestra.run.smithi064.stdout:270 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:270.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:271 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:271.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:272 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:272.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:273 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:273.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:274 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:274.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:275 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:275.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:276 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:276.out 2024-04-01T00:16:44.105 INFO:teuthology.orchestra.run.smithi064.stdout:277 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:277.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:278 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:278.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:279 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:279.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:280 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:280.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:281 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:281.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:282 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:282.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:283 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:283.out 2024-04-01T00:16:44.106 INFO:teuthology.orchestra.run.smithi064.stdout:284 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:284.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:285 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:285.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:286 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:286.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:287 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:287.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:288 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:288.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:289 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:289.out 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:290 2024-04-01T00:16:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:290.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:291 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:291.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:292 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:292.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:293 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:293.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:294 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:294.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:295 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:295.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:296 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:296.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:297 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:297.out 2024-04-01T00:16:44.108 INFO:teuthology.orchestra.run.smithi064.stdout:298 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:298.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:299 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:299.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:300 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:300.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:301 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:301.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:302 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:302.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:303 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:303.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:304 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:304.out 2024-04-01T00:16:44.109 INFO:teuthology.orchestra.run.smithi064.stdout:305 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:305.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:306 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:306.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:307 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:307.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:308 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:308.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:309 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:309.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:310 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:310.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:311 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:311.out 2024-04-01T00:16:44.110 INFO:teuthology.orchestra.run.smithi064.stdout:312 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:312.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:313 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:313.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:314 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:314.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:315 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:315.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:316 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:316.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:317 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:317.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:318 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:318.out 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:319 2024-04-01T00:16:44.111 INFO:teuthology.orchestra.run.smithi064.stdout:319.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:320 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:320.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:321 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:321.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:322 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:322.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:323 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:323.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:324 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:324.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:325 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:325.out 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:326 2024-04-01T00:16:44.112 INFO:teuthology.orchestra.run.smithi064.stdout:326.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:327 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:327.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:328 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:328.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:329 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:329.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:330 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:330.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:331 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:331.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:332 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:332.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:333 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:333.out 2024-04-01T00:16:44.113 INFO:teuthology.orchestra.run.smithi064.stdout:334 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:334.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:335 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:335.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:336 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:336.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:337 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:337.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:338 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:338.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:339 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:339.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:340 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:340.out 2024-04-01T00:16:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:341 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:341.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:342 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:342.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:343 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:343.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:344 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:344.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:345 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:345.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:346 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:346.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:347 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:347.out 2024-04-01T00:16:44.115 INFO:teuthology.orchestra.run.smithi064.stdout:348 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:348.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:349 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:349.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:350 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:350.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:351 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:351.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:352 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:352.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:353 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:353.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:354 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:354.out 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:355 2024-04-01T00:16:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:355.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:356 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:356.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:357 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:357.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:358 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:358.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:359 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:359.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:360 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:360.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:361 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:361.out 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:362 2024-04-01T00:16:44.117 INFO:teuthology.orchestra.run.smithi064.stdout:362.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:363 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:363.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:364 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:364.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:365 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:365.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:366 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:366.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:367 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:367.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:368 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:368.out 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:369 2024-04-01T00:16:44.118 INFO:teuthology.orchestra.run.smithi064.stdout:369.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:370 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:370.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:371 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:371.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:372 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:372.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:373 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:373.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:374 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:374.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:375 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:375.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:376 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:376.out 2024-04-01T00:16:44.119 INFO:teuthology.orchestra.run.smithi064.stdout:377 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:377.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:378 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:378.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:379 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:379.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:380 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:380.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:381 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:381.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:382 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:382.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:383 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:383.out 2024-04-01T00:16:44.120 INFO:teuthology.orchestra.run.smithi064.stdout:384 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:384.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:385 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:385.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:386 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:386.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:387 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:387.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:388 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:388.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:389 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:389.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:390 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:390.out 2024-04-01T00:16:44.121 INFO:teuthology.orchestra.run.smithi064.stdout:391 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:391.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:392 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:392.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:393 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:393.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:394 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:394.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:395 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:395.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:396 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:396.out 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:397 2024-04-01T00:16:44.122 INFO:teuthology.orchestra.run.smithi064.stdout:397.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:398 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:398.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:399 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:399.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:400 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:400.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:401 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:401.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:402 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:402.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:403 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:403.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:404 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:404.out 2024-04-01T00:16:44.123 INFO:teuthology.orchestra.run.smithi064.stdout:405 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:405.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:406 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:406.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:407 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:407.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:408 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:408.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:409 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:409.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:410 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:410.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:411 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:411.out 2024-04-01T00:16:44.124 INFO:teuthology.orchestra.run.smithi064.stdout:412 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:412.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:413 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:413.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:414 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:414.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:415 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:415.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:416 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:416.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:417 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:417.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:418 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:418.out 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:419 2024-04-01T00:16:44.125 INFO:teuthology.orchestra.run.smithi064.stdout:419.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:420 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:420.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:421 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:421.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:422 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:422.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:423 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:423.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:424 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:424.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:425 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:425.out 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:426 2024-04-01T00:16:44.126 INFO:teuthology.orchestra.run.smithi064.stdout:426.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:427 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:427.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:428 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:428.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:429 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:429.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:430 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:430.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:431 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:431.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:432 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:432.out 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:433 2024-04-01T00:16:44.127 INFO:teuthology.orchestra.run.smithi064.stdout:433.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:434 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:434.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:435 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:435.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:436 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:436.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:437 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:437.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:438 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:438.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:439 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:439.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:440 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:440.out 2024-04-01T00:16:44.128 INFO:teuthology.orchestra.run.smithi064.stdout:441 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:441.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:442 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:442.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:443 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:443.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:444 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:444.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:445 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:445.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:446 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:446.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:447 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:447.out 2024-04-01T00:16:44.129 INFO:teuthology.orchestra.run.smithi064.stdout:448 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:448.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:449 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:449.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:450 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:450.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:451 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:451.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:452 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:452.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:453 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:453.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:454 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:454.out 2024-04-01T00:16:44.130 INFO:teuthology.orchestra.run.smithi064.stdout:455 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:455.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:456 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:456.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:457 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:457.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:458 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:458.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:459 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:459.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:460 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:460.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:461 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:461.out 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:462 2024-04-01T00:16:44.131 INFO:teuthology.orchestra.run.smithi064.stdout:462.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:463 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:463.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:464 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:464.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:465 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:465.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:466 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:466.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:467 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:467.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:468 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:468.out 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:469 2024-04-01T00:16:44.132 INFO:teuthology.orchestra.run.smithi064.stdout:469.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:470 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:470.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:471 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:471.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:472 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:472.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:473 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:473.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:474 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:474.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:475 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:475.out 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:476 2024-04-01T00:16:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:476.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:477 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:477.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:478 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:478.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:479 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:479.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:480 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:480.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:481 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:481.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:482 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:482.out 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:483 2024-04-01T00:16:44.134 INFO:teuthology.orchestra.run.smithi064.stdout:483.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:484 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:484.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:485 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:485.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:486 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:486.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:487 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:487.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:488 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:488.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:489 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:489.out 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:490 2024-04-01T00:16:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:490.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:491 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:491.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:492 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:492.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:493 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:493.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:494 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:494.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:495 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:495.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:496 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:496.out 2024-04-01T00:16:44.136 INFO:teuthology.orchestra.run.smithi064.stdout:497 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:497.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:498 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:498.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:499 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:499.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:500 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:500.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:501 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:501.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:502 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:502.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:503 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:503.out 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:504 2024-04-01T00:16:44.137 INFO:teuthology.orchestra.run.smithi064.stdout:504.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:505 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:505.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:506 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:506.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:507 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:507.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:508 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:508.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:509 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:509.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:510 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:510.out 2024-04-01T00:16:44.138 INFO:teuthology.orchestra.run.smithi064.stdout:511 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:511.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:512 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:512.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:513 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:513.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:514 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:514.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:515 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:515.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:516 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:516.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:517 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:517.out 2024-04-01T00:16:44.139 INFO:teuthology.orchestra.run.smithi064.stdout:518 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:518.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:519 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:519.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:520 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:520.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:521 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:521.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:522 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:522.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:523 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:523.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:524 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:524.out 2024-04-01T00:16:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:525 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:525.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:526 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:526.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:527 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:527.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:528 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:528.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:529 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:529.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:530 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:530.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:531 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:531.out 2024-04-01T00:16:44.141 INFO:teuthology.orchestra.run.smithi064.stdout:532 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:532.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:533 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:533.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:534 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:534.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:535 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:535.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:536 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:536.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:537 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:537.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:538 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:538.out 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:539 2024-04-01T00:16:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:539.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:540 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:540.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:541 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:541.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:542 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:542.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:543 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:543.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:544 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:544.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:545 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:545.out 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:546 2024-04-01T00:16:44.143 INFO:teuthology.orchestra.run.smithi064.stdout:546.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:547 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:547.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:548 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:548.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:549 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:549.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:550 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:550.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:551 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:551.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:552 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:552.out 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:553 2024-04-01T00:16:44.144 INFO:teuthology.orchestra.run.smithi064.stdout:553.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:554 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:554.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:555 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:555.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:556 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:556.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:557 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:557.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:558 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:558.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:559 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:559.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:560 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:560.out 2024-04-01T00:16:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:561 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:561.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:562 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:562.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:563 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:563.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:564 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:564.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:565 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:565.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:566 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:566.out 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:567 2024-04-01T00:16:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:567.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:568 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:568.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:569 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:569.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:570 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:570.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:571 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:571.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:572 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:572.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:573 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:573.out 2024-04-01T00:16:44.147 INFO:teuthology.orchestra.run.smithi064.stdout:574 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:574.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:575 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:575.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:576 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:576.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:577 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:577.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:578 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:578.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:579 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:579.out 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:580 2024-04-01T00:16:44.148 INFO:teuthology.orchestra.run.smithi064.stdout:580.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:581 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:581.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:582 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:582.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:583 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:583.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:584 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:584.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:585 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:585.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:586 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:586.out 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:587 2024-04-01T00:16:44.149 INFO:teuthology.orchestra.run.smithi064.stdout:587.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:588 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:588.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:589 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:589.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:590 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:590.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:591 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:591.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:592 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:592.out 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:593 2024-04-01T00:16:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:593.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:594 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:594.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:595 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:595.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:596 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:596.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:597 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:597.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:598 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:598.out 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:599 2024-04-01T00:16:44.151 INFO:teuthology.orchestra.run.smithi064.stdout:599.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:600 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:600.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:601 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:601.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:602 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:602.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:603 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:603.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:604 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:604.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:605 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:605.out 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:606 2024-04-01T00:16:44.152 INFO:teuthology.orchestra.run.smithi064.stdout:606.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:607 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:607.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:608 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:608.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:609 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:609.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:610 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:610.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:611 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:611.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:612 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:612.out 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:613 2024-04-01T00:16:44.153 INFO:teuthology.orchestra.run.smithi064.stdout:613.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:614 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:614.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:615 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:615.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:616 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:616.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:617 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:617.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:618 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:618.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:619 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:619.out 2024-04-01T00:16:44.154 INFO:teuthology.orchestra.run.smithi064.stdout:620 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:620.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:621 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:621.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:622 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:622.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:623 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:623.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:624 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:624.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:625 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:625.out 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:626 2024-04-01T00:16:44.155 INFO:teuthology.orchestra.run.smithi064.stdout:626.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:627 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:627.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:628 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:628.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:629 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:629.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:630 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:630.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:631 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:631.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:632 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:632.out 2024-04-01T00:16:44.156 INFO:teuthology.orchestra.run.smithi064.stdout:633 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:633.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:634 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:634.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:635 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:635.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:636 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:636.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:637 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:637.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:638 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:638.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:639 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:639.out 2024-04-01T00:16:44.157 INFO:teuthology.orchestra.run.smithi064.stdout:640 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:640.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:641 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:641.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:642 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:642.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:643 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:643.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:644 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:644.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:645 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:645.out 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:646 2024-04-01T00:16:44.158 INFO:teuthology.orchestra.run.smithi064.stdout:646.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:647 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:647.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:648 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:648.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:649 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:649.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:650 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:650.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:651 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:651.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:652 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:652.out 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:653 2024-04-01T00:16:44.159 INFO:teuthology.orchestra.run.smithi064.stdout:653.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:654 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:654.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:655 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:655.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:656 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:656.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:657 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:657.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:658 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:658.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:659 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:659.out 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:660 2024-04-01T00:16:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:660.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:661 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:661.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:662 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:662.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:663 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:663.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:664 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:664.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:665 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:665.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:666 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:666.out 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:667 2024-04-01T00:16:44.161 INFO:teuthology.orchestra.run.smithi064.stdout:667.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:668 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:668.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:669 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:669.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:670 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:670.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:671 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:671.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:672 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:672.out 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:673 2024-04-01T00:16:44.162 INFO:teuthology.orchestra.run.smithi064.stdout:673.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:674 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:674.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:675 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:675.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:676 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:676.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:677 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:677.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:678 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:678.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:679 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:679.out 2024-04-01T00:16:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:680 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:680.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:681 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:681.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:682 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:682.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:683 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:683.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:684 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:684.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:685 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:685.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:686 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:686.out 2024-04-01T00:16:44.164 INFO:teuthology.orchestra.run.smithi064.stdout:687 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:687.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:688 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:688.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:689 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:689.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:690 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:690.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:691 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:691.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:692 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:692.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:693 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:693.out 2024-04-01T00:16:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:694 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:694.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:695 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:695.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:696 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:696.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:697 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:697.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:698 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:698.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:699 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:699.out 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:700 2024-04-01T00:16:44.166 INFO:teuthology.orchestra.run.smithi064.stdout:700.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:701 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:701.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:702 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:702.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:703 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:703.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:704 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:704.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:705 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:705.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:706 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:706.out 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:707 2024-04-01T00:16:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:707.out 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:708 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:708.out 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:709 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:709.out 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:710 2024-04-01T00:16:44.168 INFO:teuthology.orchestra.run.smithi064.stdout:710.out 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:711 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:711.out 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:712 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:712.out 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:713 2024-04-01T00:16:44.169 INFO:teuthology.orchestra.run.smithi064.stdout:713.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:714 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:714.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:715 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:715.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:716 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:716.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:717 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:717.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:718 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:718.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:719 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:719.out 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:720 2024-04-01T00:16:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:720.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:721 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:721.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:722 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:722.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:723 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:723.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:724 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:724.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:725 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:725.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:726 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:726.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:727 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:727.out 2024-04-01T00:16:44.171 INFO:teuthology.orchestra.run.smithi064.stdout:728 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:728.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:729 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:729.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:730 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:730.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:731 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:731.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:732 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:732.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:733 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:733.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:734 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:734.out 2024-04-01T00:16:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:735 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:735.out 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:736 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:736.out 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:737 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:737.out 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:738 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:738.out 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:739 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:739.out 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:group.list 2024-04-01T00:16:44.173 INFO:teuthology.orchestra.run.smithi064.stdout:Makefile 2024-04-01T00:16:44.174 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-04-01T00:16:44.329 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:44.332 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:44.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:46.176 INFO:teuthology.orchestra.run.smithi064.stdout:generic/001 2s 2024-04-01T00:16:46.181 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/001 2024-04-01T00:16:46.181 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:16:46.183 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:46.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52676]: fuse finished with error 0 and tester_r 0 2024-04-01T00:16:46.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:16:46.192 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-04-01T00:16:46.362 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:46.366 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:46.366 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:46.764 INFO:teuthology.orchestra.run.smithi064.stdout:generic/002 0s 2024-04-01T00:16:46.769 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/002 2024-04-01T00:16:46.769 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:16:46.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:46.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:16:46.780 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-04-01T00:16:46.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:16:46.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:16:46.969 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:46.973 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:46.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:47.090 INFO:teuthology.orchestra.run.smithi064.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:16:47.094 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/003 2024-04-01T00:16:47.094 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/003 2024-04-01T00:16:47.094 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:16:47.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:47.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:16:47.104 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-04-01T00:16:47.318 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:47.322 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:47.322 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:47.454 INFO:teuthology.orchestra.run.smithi064.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-04-01T00:16:47.457 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/004 2024-04-01T00:16:47.457 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/004 2024-04-01T00:16:47.457 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:16:47.460 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:47.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:16:47.468 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-04-01T00:16:47.650 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:47.651 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:47.651 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:generic/005 1s 2024-04-01T00:16:48.184 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/005 2024-04-01T00:16:48.184 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:16:48.186 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:16:48.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:16:48.195 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-04-01T00:16:48.408 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:16:48.412 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:16:48.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:17:17.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:17:17.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:17:24.149 INFO:teuthology.orchestra.run.smithi064.stdout:generic/006 36s 2024-04-01T00:17:24.155 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/006 2024-04-01T00:17:24.155 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:17:24.157 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:17:24.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:17:24.214 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-04-01T00:17:24.385 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:17:24.389 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:17:24.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:17:47.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:17:47.207 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:18:17.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:18:17.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:18:36.940 INFO:teuthology.orchestra.run.smithi064.stdout:generic/007 72s 2024-04-01T00:18:36.944 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/007 2024-04-01T00:18:36.944 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:18:36.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:36.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:18:36.955 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-04-01T00:18:37.161 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:18:37.165 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:18:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:38.011 INFO:teuthology.orchestra.run.smithi064.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-01T00:18:38.014 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/008 2024-04-01T00:18:38.014 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/008 2024-04-01T00:18:38.014 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:18:38.016 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:38.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:18:38.024 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-04-01T00:18:38.204 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:18:38.209 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:18:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:38.997 INFO:teuthology.orchestra.run.smithi064.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-01T00:18:39.002 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/009 2024-04-01T00:18:39.002 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/009 2024-04-01T00:18:39.002 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:18:39.003 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:39.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:18:39.011 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-04-01T00:18:39.191 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:18:39.195 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:18:39.195 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:39.815 INFO:teuthology.orchestra.run.smithi064.stdout:generic/010 0s 2024-04-01T00:18:39.888 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/010 2024-04-01T00:18:39.888 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:18:39.890 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:39.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:18:39.903 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-04-01T00:18:40.091 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:18:40.095 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:18:40.095 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:18:47.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:18:47.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:19:17.563 INFO:teuthology.orchestra.run.smithi064.stdout:generic/011 37s 2024-04-01T00:19:17.568 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/011 2024-04-01T00:19:17.568 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:19:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:19:17.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:19:17.580 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-04-01T00:19:17.782 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:19:17.786 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:19:17.786 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:19:18.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:19:18.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:19:18.782 INFO:teuthology.orchestra.run.smithi064.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:19:18.785 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/012 2024-04-01T00:19:18.786 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/012 2024-04-01T00:19:18.786 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:19:18.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:19:18.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:19:18.810 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-04-01T00:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:19:19.013 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:19:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:19:48.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:19:48.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:20:18.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:20:18.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:20:18.906 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T00:20:18.905+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 44807) UID: 0 2024-04-01T00:20:18.906 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T00:20:18.905+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 44807) UID: 0 2024-04-01T00:20:36.801 INFO:teuthology.orchestra.run.smithi064.stdout:generic/013 77s 2024-04-01T00:20:36.807 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/013 2024-04-01T00:20:36.808 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:36.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:36.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:36.822 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-04-01T00:20:37.057 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:37.262 INFO:teuthology.orchestra.run.smithi064.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-04-01T00:20:37.266 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/014 2024-04-01T00:20:37.266 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/014 2024-04-01T00:20:37.266 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:37.268 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:37.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:37.281 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-04-01T00:20:37.514 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:37.519 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:37.519 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:37.674 INFO:teuthology.orchestra.run.smithi064.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:20:37.678 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/015 2024-04-01T00:20:37.678 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/015 2024-04-01T00:20:37.678 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:37.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:37.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:37.693 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-04-01T00:20:37.949 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:37.955 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:37.955 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:39.355 INFO:teuthology.orchestra.run.smithi064.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:20:39.359 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/016 2024-04-01T00:20:39.359 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/016 2024-04-01T00:20:39.359 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:39.362 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:39.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:39.375 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-04-01T00:20:39.665 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:39.671 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:39.671 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:39.829 INFO:teuthology.orchestra.run.smithi064.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:20:39.833 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/017 2024-04-01T00:20:39.833 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/017 2024-04-01T00:20:39.833 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:39.836 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:39.846 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:39.847 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-04-01T00:20:40.076 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:40.081 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:40.081 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/018 2024-04-01T00:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/018 2024-04-01T00:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:40.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:40.252 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-04-01T00:20:40.494 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:40.499 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:40.499 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:40.644 INFO:teuthology.orchestra.run.smithi064.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:20:40.648 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/019 2024-04-01T00:20:40.648 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/019 2024-04-01T00:20:40.649 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:40.651 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:40.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:40.662 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-04-01T00:20:40.904 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:40.909 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:40.909 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:50.484 INFO:teuthology.orchestra.run.smithi064.stdout:generic/020 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/020.out.bad) 2024-04-01T00:20:50.486 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/020.out 2024-04-01 00:14:56.993908522 +0000 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/020.out.bad 2024-04-01 00:20:50.481910938 +0000 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -47,9 +47,13 @@ 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: user.snrub="fish2\012" 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: *** really long value 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: -0000000 00 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: -* 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: -ATTRSIZE 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: +attr_set: No space left on device 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: +Could not set "long_attr" for 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: +attr_get: No data available 2024-04-01T00:20:50.487 INFO:teuthology.orchestra.run.smithi064.stdout: +Could not get "long_attr" for 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: +0000000 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: +attr_remove: No data available 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: +Could not remove "long_attr" for 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: *** set/get/remove really long names (expect failure) 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: attr_set: Invalid argument 2024-04-01T00:20:50.488 INFO:teuthology.orchestra.run.smithi064.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-04-01T00:20:50.490 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/020 2024-04-01T00:20:50.491 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/020 2024-04-01T00:20:50.491 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T00:20:50.493 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:50.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T00:20:50.508 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T00:20:50.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T00:20:50.509 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-04-01T00:20:50.792 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:50.797 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:50.797 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:52.134 INFO:teuthology.orchestra.run.smithi064.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:20:52.139 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/021 2024-04-01T00:20:52.139 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/021 2024-04-01T00:20:52.139 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:52.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:52.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:52.153 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-04-01T00:20:52.429 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:52.435 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:52.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:53.789 INFO:teuthology.orchestra.run.smithi064.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:20:53.794 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/022 2024-04-01T00:20:53.794 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/022 2024-04-01T00:20:53.794 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:53.797 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:53.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:53.811 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-04-01T00:20:54.097 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:54.101 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:54.101 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:55.751 INFO:teuthology.orchestra.run.smithi064.stdout:generic/023 1s 2024-04-01T00:20:55.758 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/023 2024-04-01T00:20:55.759 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:55.762 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:55.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:55.774 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-04-01T00:20:56.072 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:56.077 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:56.077 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:56.260 INFO:teuthology.orchestra.run.smithi064.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-04-01T00:20:56.264 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/024 2024-04-01T00:20:56.264 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/024 2024-04-01T00:20:56.264 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:56.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:56.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:56.277 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-04-01T00:20:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:56.580 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:56.580 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:56.775 INFO:teuthology.orchestra.run.smithi064.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-04-01T00:20:56.780 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/025 2024-04-01T00:20:56.780 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/025 2024-04-01T00:20:56.780 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:56.782 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:56.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:56.793 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-04-01T00:20:57.084 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:57.090 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:57.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:57.297 INFO:teuthology.orchestra.run.smithi064.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T00:20:57.301 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/026 2024-04-01T00:20:57.301 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/026 2024-04-01T00:20:57.301 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:57.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:57.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:57.314 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-04-01T00:20:57.549 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:57.555 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:57.555 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:57.702 INFO:teuthology.orchestra.run.smithi064.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:20:57.706 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/027 2024-04-01T00:20:57.706 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/027 2024-04-01T00:20:57.706 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:20:57.710 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:20:57.720 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:20:57.720 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-04-01T00:20:57.956 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:20:57.962 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:20:57.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.136 INFO:teuthology.orchestra.run.smithi064.stdout:generic/028 6s 2024-04-01T00:21:03.143 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/028 2024-04-01T00:21:03.143 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:03.158 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-04-01T00:21:03.402 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:03.407 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.555 INFO:teuthology.orchestra.run.smithi064.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:03.560 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/029 2024-04-01T00:21:03.560 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/029 2024-04-01T00:21:03.560 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:03.576 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-04-01T00:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.966 INFO:teuthology.orchestra.run.smithi064.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:03.971 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/030 2024-04-01T00:21:03.971 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/030 2024-04-01T00:21:03.971 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:03.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:03.987 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-04-01T00:21:04.230 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:04.235 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:04.376 INFO:teuthology.orchestra.run.smithi064.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:04.380 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/031 2024-04-01T00:21:04.380 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/031 2024-04-01T00:21:04.380 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:04.383 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:04.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:04.394 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-04-01T00:21:04.626 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:04.631 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:04.631 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:04.775 INFO:teuthology.orchestra.run.smithi064.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:04.779 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/032 2024-04-01T00:21:04.779 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/032 2024-04-01T00:21:04.779 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:04.782 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:04.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:04.793 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-04-01T00:21:05.016 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:05.023 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:05.175 INFO:teuthology.orchestra.run.smithi064.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:05.179 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/033 2024-04-01T00:21:05.180 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/033 2024-04-01T00:21:05.180 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:05.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:05.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:05.192 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-04-01T00:21:05.421 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:05.426 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:05.426 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:05.595 INFO:teuthology.orchestra.run.smithi064.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:05.600 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/034 2024-04-01T00:21:05.600 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/034 2024-04-01T00:21:05.600 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:05.603 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:05.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:05.614 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-04-01T00:21:05.850 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:05.854 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:05.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:06.091 INFO:teuthology.orchestra.run.smithi064.stdout:generic/035 1s 2024-04-01T00:21:06.098 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/035 2024-04-01T00:21:06.099 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:06.101 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:06.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:06.111 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-04-01T00:21:06.422 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:06.427 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:06.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:16.740 INFO:teuthology.orchestra.run.smithi064.stdout:generic/036 10s 2024-04-01T00:21:16.791 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/036 2024-04-01T00:21:16.791 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:16.793 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:16.804 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:16.805 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-04-01T00:21:17.108 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:17.112 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:17.262 INFO:teuthology.orchestra.run.smithi064.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:17.266 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/037 2024-04-01T00:21:17.266 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/037 2024-04-01T00:21:17.266 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:17.270 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:17.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:17.281 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-04-01T00:21:17.510 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:17.516 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:17.516 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:17.671 INFO:teuthology.orchestra.run.smithi064.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:17.676 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/038 2024-04-01T00:21:17.676 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/038 2024-04-01T00:21:17.676 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:17.679 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:17.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:17.688 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-04-01T00:21:17.926 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:17.932 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:17.932 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/039 2024-04-01T00:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/039 2024-04-01T00:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:18.111 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-04-01T00:21:18.336 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:18.341 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:18.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.498 INFO:teuthology.orchestra.run.smithi064.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:18.502 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/040 2024-04-01T00:21:18.502 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/040 2024-04-01T00:21:18.502 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:18.516 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-04-01T00:21:18.755 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:18.760 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:18.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.927 INFO:teuthology.orchestra.run.smithi064.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:18.931 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/041 2024-04-01T00:21:18.932 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/041 2024-04-01T00:21:18.932 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:18.935 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:18.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:18.945 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-04-01T00:21:19.177 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:19.182 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:19.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:19.328 INFO:teuthology.orchestra.run.smithi064.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:19.331 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/042 2024-04-01T00:21:19.332 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/042 2024-04-01T00:21:19.332 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:19.335 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:19.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:19.345 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-04-01T00:21:19.582 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:19.588 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:19.732 INFO:teuthology.orchestra.run.smithi064.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:19.737 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/043 2024-04-01T00:21:19.737 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/043 2024-04-01T00:21:19.737 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:19.739 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:19.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:19.752 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-04-01T00:21:19.991 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:19.996 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:19.996 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:20.142 INFO:teuthology.orchestra.run.smithi064.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:20.146 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/044 2024-04-01T00:21:20.146 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/044 2024-04-01T00:21:20.146 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:20.149 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:20.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:20.160 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-04-01T00:21:20.395 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:20.401 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:20.401 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:20.576 INFO:teuthology.orchestra.run.smithi064.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:20.581 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/045 2024-04-01T00:21:20.581 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/045 2024-04-01T00:21:20.581 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:20.584 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:20.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:20.593 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-04-01T00:21:20.860 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:20.865 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.022 INFO:teuthology.orchestra.run.smithi064.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:21.027 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/046 2024-04-01T00:21:21.027 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/046 2024-04-01T00:21:21.027 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:21.042 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-04-01T00:21:21.295 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:21.300 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.466 INFO:teuthology.orchestra.run.smithi064.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:21.472 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/047 2024-04-01T00:21:21.472 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/047 2024-04-01T00:21:21.472 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:21.476 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:21.488 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-04-01T00:21:21.744 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:21.749 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:21.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.920 INFO:teuthology.orchestra.run.smithi064.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:21.925 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/048 2024-04-01T00:21:21.925 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/048 2024-04-01T00:21:21.925 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:21.928 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:21.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:21.939 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-04-01T00:21:22.184 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:22.189 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:22.189 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:22.340 INFO:teuthology.orchestra.run.smithi064.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/049 2024-04-01T00:21:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/049 2024-04-01T00:21:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:22.347 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:22.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:22.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-04-01T00:21:22.588 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:22.593 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:22.747 INFO:teuthology.orchestra.run.smithi064.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:22.752 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/050 2024-04-01T00:21:22.752 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/050 2024-04-01T00:21:22.752 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:22.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:22.765 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:22.765 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-04-01T00:21:22.994 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:23.001 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:23.001 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:23.152 INFO:teuthology.orchestra.run.smithi064.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:23.156 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/051 2024-04-01T00:21:23.156 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/051 2024-04-01T00:21:23.156 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:23.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:23.179 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:23.180 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-04-01T00:21:23.403 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:23.407 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:23.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:23.558 INFO:teuthology.orchestra.run.smithi064.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:23.561 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/052 2024-04-01T00:21:23.562 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/052 2024-04-01T00:21:23.562 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:23.564 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:23.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:23.575 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-04-01T00:21:23.805 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:23.809 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:23.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:23.968 INFO:teuthology.orchestra.run.smithi064.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:24.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/053 2024-04-01T00:21:24.009 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/053 2024-04-01T00:21:24.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:24.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:24.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:24.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-04-01T00:21:24.239 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:24.247 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:24.247 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/054 2024-04-01T00:21:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/054 2024-04-01T00:21:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:24.406 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:24.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:24.416 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-04-01T00:21:24.644 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:24.649 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:24.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:24.813 INFO:teuthology.orchestra.run.smithi064.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:24.818 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/055 2024-04-01T00:21:24.818 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/055 2024-04-01T00:21:24.818 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:24.821 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:24.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:24.831 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-04-01T00:21:25.065 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:25.070 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:25.233 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/056 2024-04-01T00:21:25.233 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/056 2024-04-01T00:21:25.233 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:25.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:25.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:25.245 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-04-01T00:21:25.474 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:25.479 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:25.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:25.633 INFO:teuthology.orchestra.run.smithi064.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:25.637 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/057 2024-04-01T00:21:25.637 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/057 2024-04-01T00:21:25.637 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:25.640 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:25.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:25.651 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-04-01T00:21:25.896 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:25.900 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:25.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:21:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/058 2024-04-01T00:21:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/058 2024-04-01T00:21:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:27.239 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:27.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:27.251 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-04-01T00:21:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:27.719 INFO:teuthology.orchestra.run.smithi064.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:27.723 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/059 2024-04-01T00:21:27.724 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/059 2024-04-01T00:21:27.724 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:27.736 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:27.737 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-04-01T00:21:27.965 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:27.969 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:29.314 INFO:teuthology.orchestra.run.smithi064.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:21:29.318 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/060 2024-04-01T00:21:29.318 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/060 2024-04-01T00:21:29.318 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:29.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:29.347 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-04-01T00:21:29.645 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:29.652 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:29.653 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:30.984 INFO:teuthology.orchestra.run.smithi064.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:21:30.988 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/061 2024-04-01T00:21:30.988 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/061 2024-04-01T00:21:30.988 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:30.991 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:31.021 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:31.021 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-04-01T00:21:31.320 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:31.325 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:31.325 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:31.479 INFO:teuthology.orchestra.run.smithi064.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:31.486 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/062 2024-04-01T00:21:31.487 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/062 2024-04-01T00:21:31.487 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:31.490 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:31.501 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:31.501 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-04-01T00:21:31.739 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:31.744 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:33.076 INFO:teuthology.orchestra.run.smithi064.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T00:21:33.080 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/063 2024-04-01T00:21:33.080 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/063 2024-04-01T00:21:33.080 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:33.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:33.110 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-04-01T00:21:33.419 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:33.425 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:33.574 INFO:teuthology.orchestra.run.smithi064.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/064 2024-04-01T00:21:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/064 2024-04-01T00:21:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:33.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:33.596 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-04-01T00:21:33.829 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:33.836 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:33.992 INFO:teuthology.orchestra.run.smithi064.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:33.997 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/065 2024-04-01T00:21:33.997 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/065 2024-04-01T00:21:33.997 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:33.999 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:34.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:34.010 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-04-01T00:21:34.254 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:34.258 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:34.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:34.419 INFO:teuthology.orchestra.run.smithi064.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:34.423 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/066 2024-04-01T00:21:34.423 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/066 2024-04-01T00:21:34.423 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:34.426 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:34.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:34.438 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-04-01T00:21:34.670 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:34.675 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:34.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:34.867 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/067 2024-04-01T00:21:34.867 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/067 2024-04-01T00:21:34.867 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:34.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:34.882 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-04-01T00:21:35.121 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:35.126 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:35.126 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:35.279 INFO:teuthology.orchestra.run.smithi064.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:35.283 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/068 2024-04-01T00:21:35.283 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/068 2024-04-01T00:21:35.283 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:35.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:35.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:35.296 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-04-01T00:21:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:35.539 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:35.539 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:35.692 INFO:teuthology.orchestra.run.smithi064.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:21:35.696 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/069 2024-04-01T00:21:35.696 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/069 2024-04-01T00:21:35.697 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:21:35.700 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:21:35.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:21:35.713 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-04-01T00:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:06.272 INFO:teuthology.orchestra.run.smithi064.stdout:generic/070 91s 2024-04-01T00:23:06.277 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/070 2024-04-01T00:23:06.277 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:23:06.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:06.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:23:06.293 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-04-01T00:23:06.658 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:23:06.664 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:23:06.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:06.822 INFO:teuthology.orchestra.run.smithi064.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:23:06.827 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/071 2024-04-01T00:23:06.827 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/071 2024-04-01T00:23:06.827 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:23:06.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:06.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:23:06.842 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-04-01T00:23:07.128 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:23:07.134 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:23:07.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:08.480 INFO:teuthology.orchestra.run.smithi064.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-01T00:23:08.482 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/072 2024-04-01T00:23:08.484 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/072 2024-04-01T00:23:08.484 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:23:08.486 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:08.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:23:08.500 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-04-01T00:23:08.773 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:23:08.778 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:23:08.778 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:08.948 INFO:teuthology.orchestra.run.smithi064.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:23:08.951 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/073 2024-04-01T00:23:08.951 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/073 2024-04-01T00:23:08.951 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:23:08.954 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:08.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:23:08.965 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-04-01T00:23:09.194 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:23:09.200 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:23:09.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:23:28.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:23:28.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:23:59.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:23:59.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:24:29.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:24:29.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:24:59.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:24:59.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:25:29.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:25:29.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:25:59.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:25:59.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:26:29.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:26:29.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:26:59.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:26:59.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:27:29.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:27:29.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:27:59.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:27:59.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:28:30.102 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:28:30.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:29:00.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:29:00.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:29:31.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:29:31.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:30:01.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:30:01.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:30:31.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:30:31.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:31:01.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:31:01.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:31:31.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:31:31.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:32:02.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:32:02.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:32:32.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:32:32.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:33:02.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:33:02.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:33:32.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:33:32.756 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:34:02.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:34:02.886 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:34:33.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:34:33.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:35:03.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:35:03.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:35:33.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:35:33.395 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:36:03.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:36:03.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:36:33.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:36:33.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:37:04.255 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:37:04.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:37:34.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:37:34.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:38:04.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:38:04.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:38:34.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:38:34.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:39:04.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:39:04.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:39:34.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:39:34.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:40:04.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:40:04.800 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:40:34.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:40:34.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:41:05.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:41:05.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:41:35.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:41:35.333 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:42:05.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:42:05.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:42:35.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:42:35.480 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:43:05.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:43:05.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:43:35.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:43:35.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:44:05.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:44:05.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:44:36.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:44:36.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:45:06.085 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:45:06.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:45:36.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:45:36.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:46:06.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:46:06.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:46:36.644 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:46:36.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:47:06.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:47:06.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:47:37.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:47:37.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:48:07.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:48:07.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:48:37.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:48:37.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:49:07.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:49:07.358 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:49:37.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:49:37.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:49:37.494 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T00:49:37.493+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46110) UID: 0 2024-04-01T00:49:37.561 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T00:49:37.493+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46110) UID: 0 2024-04-01T00:49:37.562 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T00:49:37.494+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46110) UID: 0 2024-04-01T00:49:37.562 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T00:49:37.493+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46110) UID: 0 2024-04-01T00:50:29.677 INFO:teuthology.orchestra.run.smithi064.stdout:generic/074 1640s 2024-04-01T00:50:29.711 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/074 2024-04-01T00:50:29.711 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:50:29.714 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:50:29.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:50:29.725 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-04-01T00:50:30.396 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:50:30.399 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:50:30.399 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:12.852 INFO:teuthology.orchestra.run.smithi064.stdout:generic/075 42s 2024-04-01T00:51:12.856 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/075 2024-04-01T00:51:12.856 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:12.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:12.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:12.868 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-04-01T00:51:13.389 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:13.392 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:13.392 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:13.517 INFO:teuthology.orchestra.run.smithi064.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:13.520 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/076 2024-04-01T00:51:13.520 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/076 2024-04-01T00:51:13.520 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:13.522 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:13.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:13.531 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-04-01T00:51:13.698 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:13.701 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:13.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:14.387 INFO:teuthology.orchestra.run.smithi064.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:14.391 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/077 2024-04-01T00:51:14.391 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/077 2024-04-01T00:51:14.391 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:14.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:14.401 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-04-01T00:51:14.571 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:14.574 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:14.736 INFO:teuthology.orchestra.run.smithi064.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-04-01T00:51:14.739 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/078 2024-04-01T00:51:14.739 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/078 2024-04-01T00:51:14.739 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:14.741 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:14.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:14.750 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-04-01T00:51:15.242 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:15.246 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:15.403 INFO:teuthology.orchestra.run.smithi064.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-04-01T00:51:15.406 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/079 2024-04-01T00:51:15.406 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/079 2024-04-01T00:51:15.406 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:15.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:15.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:15.417 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-04-01T00:51:15.587 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:15.590 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:15.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:17.759 INFO:teuthology.orchestra.run.smithi064.stdout:generic/080 2s 2024-04-01T00:51:17.765 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/080 2024-04-01T00:51:17.765 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:17.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:17.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:17.777 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-04-01T00:51:17.957 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:17.961 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:17.961 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.080 INFO:teuthology.orchestra.run.smithi064.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:18.084 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/081 2024-04-01T00:51:18.084 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/081 2024-04-01T00:51:18.084 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:18.094 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-04-01T00:51:18.263 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.394 INFO:teuthology.orchestra.run.smithi064.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:18.397 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/082 2024-04-01T00:51:18.397 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/082 2024-04-01T00:51:18.397 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:18.399 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:18.408 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-04-01T00:51:18.578 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:18.582 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.710 INFO:teuthology.orchestra.run.smithi064.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/083 2024-04-01T00:51:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/083 2024-04-01T00:51:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:18.716 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:18.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:18.724 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-04-01T00:51:18.902 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:18.905 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:18.906 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:19.019 INFO:teuthology.orchestra.run.smithi064.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/084 2024-04-01T00:51:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/084 2024-04-01T00:51:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:19.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:19.033 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-04-01T00:51:19.206 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:19.208 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:19.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:19.373 INFO:teuthology.orchestra.run.smithi064.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T00:51:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/085 2024-04-01T00:51:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/085 2024-04-01T00:51:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:19.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:19.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:19.397 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-04-01T00:51:19.571 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:19.575 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:19.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:24.878 INFO:teuthology.orchestra.run.smithi064.stdout:generic/086 5s 2024-04-01T00:51:24.919 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/086 2024-04-01T00:51:24.919 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:25.063 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:25.063 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-04-01T00:51:26.199 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:26.204 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:26.204 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:26.565 INFO:teuthology.orchestra.run.smithi064.stdout:generic/087 0s 2024-04-01T00:51:26.570 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/087 2024-04-01T00:51:26.571 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:26.573 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:26.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:26.581 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-04-01T00:51:27.079 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:27.083 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:generic/088 0s 2024-04-01T00:51:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/088 2024-04-01T00:51:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T00:51:27.266 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:51:27.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T00:51:27.274 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-04-01T00:51:27.767 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T00:51:27.770 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T00:51:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T00:52:32.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:52:32.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:53:02.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:53:02.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:53:32.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:53:32.484 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:54:02.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:54:02.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:54:02.862 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T00:54:02.863+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157788) UID: 0 2024-04-01T00:54:02.863 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T00:54:02.863+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157788) UID: 0 2024-04-01T00:54:02.863 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T00:54:02.863+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157788) UID: 0 2024-04-01T00:54:02.863 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T00:54:02.863+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157788) UID: 0 2024-04-01T00:56:49.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:56:49.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T00:56:49.612 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T00:56:49.613+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46209) UID: 0 2024-04-01T00:56:49.613 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T00:56:49.613+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46209) UID: 0 2024-04-01T01:01:39.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:01:39.557 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:01:39.747 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T01:01:39.748+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46250) UID: 0 2024-04-01T01:01:39.748 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T01:01:39.748+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46250) UID: 0 2024-04-01T01:05:06.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:05:06.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:05:36.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:05:36.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:06:04.095 INFO:teuthology.orchestra.run.smithi064.stdout:generic/089 877s 2024-04-01T01:06:04.099 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/089 2024-04-01T01:06:04.100 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:06:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:06:04.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:06:04.114 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-04-01T01:06:04.618 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:06:04.623 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:06:04.623 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:06:04.785 INFO:teuthology.orchestra.run.smithi064.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:06:04.789 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/090 2024-04-01T01:06:04.789 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/090 2024-04-01T01:06:04.789 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:06:04.791 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:06:04.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:06:04.801 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-04-01T01:06:04.978 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:06:04.982 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:06:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:06:06.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:06:06.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:06:06.229 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T01:06:06.230+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46310) UID: 0 2024-04-01T01:06:06.230 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T01:06:06.230+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46310) UID: 0 2024-04-01T01:08:10.686 INFO:teuthology.orchestra.run.smithi064.stdout:generic/091 126s 2024-04-01T01:08:10.691 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/091 2024-04-01T01:08:10.691 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:10.693 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:10.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:10.702 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-04-01T01:08:11.199 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:11.202 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:11.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:11.374 INFO:teuthology.orchestra.run.smithi064.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T01:08:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/092 2024-04-01T01:08:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/092 2024-04-01T01:08:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:11.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:11.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:11.388 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-04-01T01:08:11.853 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:11.857 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:11.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:12.603 INFO:teuthology.orchestra.run.smithi064.stdout:generic/093 1s 2024-04-01T01:08:12.607 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/093 2024-04-01T01:08:12.608 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:12.610 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:12.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:12.619 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-04-01T01:08:13.134 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.254 INFO:teuthology.orchestra.run.smithi064.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:13.257 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/094 2024-04-01T01:08:13.257 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/094 2024-04-01T01:08:13.257 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:13.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:13.268 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-04-01T01:08:13.444 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:13.448 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.579 INFO:teuthology.orchestra.run.smithi064.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:13.582 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/095 2024-04-01T01:08:13.582 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/095 2024-04-01T01:08:13.583 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:13.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:13.593 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-04-01T01:08:13.762 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:13.766 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:13.766 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.890 INFO:teuthology.orchestra.run.smithi064.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:13.893 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/096 2024-04-01T01:08:13.894 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/096 2024-04-01T01:08:13.894 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:13.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:13.904 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-04-01T01:08:14.086 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:14.090 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:14.787 INFO:teuthology.orchestra.run.smithi064.stdout:generic/097 0s 2024-04-01T01:08:14.792 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/097 2024-04-01T01:08:14.792 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:14.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:14.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:14.802 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-04-01T01:08:14.976 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:14.980 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:14.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:15.094 INFO:teuthology.orchestra.run.smithi064.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/098 2024-04-01T01:08:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/098 2024-04-01T01:08:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:15.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:15.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:15.108 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-01T01:08:15.281 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:15.285 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:15.285 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:15.439 INFO:teuthology.orchestra.run.smithi064.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T01:08:15.442 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/099 2024-04-01T01:08:15.442 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/099 2024-04-01T01:08:15.442 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:15.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:15.452 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-04-01T01:08:15.625 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:15.629 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:15.629 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:41.840 INFO:teuthology.orchestra.run.smithi064.stdout:generic/100 26s 2024-04-01T01:08:41.845 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/100 2024-04-01T01:08:41.845 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:41.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:41.862 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-04-01T01:08:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:42.192 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:42.422 INFO:teuthology.orchestra.run.smithi064.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:42.426 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/101 2024-04-01T01:08:42.426 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/101 2024-04-01T01:08:42.426 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:42.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:42.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-04-01T01:08:42.626 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:42.630 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:42.754 INFO:teuthology.orchestra.run.smithi064.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:42.757 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/102 2024-04-01T01:08:42.757 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/102 2024-04-01T01:08:42.758 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:42.779 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:42.779 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-04-01T01:08:42.959 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:42.963 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.089 INFO:teuthology.orchestra.run.smithi064.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:43.093 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/103 2024-04-01T01:08:43.093 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/103 2024-04-01T01:08:43.093 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:43.095 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:43.104 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-04-01T01:08:43.278 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:43.282 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.405 INFO:teuthology.orchestra.run.smithi064.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:43.409 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/104 2024-04-01T01:08:43.409 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/104 2024-04-01T01:08:43.409 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:43.419 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-04-01T01:08:43.591 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:43.595 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:43.595 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.707 INFO:teuthology.orchestra.run.smithi064.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:43.711 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/105 2024-04-01T01:08:43.711 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/105 2024-04-01T01:08:43.711 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:43.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:43.722 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-04-01T01:08:43.905 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:43.909 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.037 INFO:teuthology.orchestra.run.smithi064.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:44.040 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/106 2024-04-01T01:08:44.040 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/106 2024-04-01T01:08:44.040 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:44.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:44.051 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-04-01T01:08:44.230 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:44.234 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:44.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.366 INFO:teuthology.orchestra.run.smithi064.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:44.369 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/107 2024-04-01T01:08:44.369 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/107 2024-04-01T01:08:44.369 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:44.371 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:44.379 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-04-01T01:08:44.552 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:44.555 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:44.556 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.844 INFO:teuthology.orchestra.run.smithi064.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:44.847 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/108 2024-04-01T01:08:44.847 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/108 2024-04-01T01:08:44.847 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:44.849 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:44.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:44.858 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-04-01T01:08:45.030 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:45.034 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:45.034 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:45.154 INFO:teuthology.orchestra.run.smithi064.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:08:45.158 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/109 2024-04-01T01:08:45.158 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/109 2024-04-01T01:08:45.158 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:45.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:45.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:45.168 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-04-01T01:08:45.338 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:45.342 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:45.342 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:45.503 INFO:teuthology.orchestra.run.smithi064.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:08:45.506 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/110 2024-04-01T01:08:45.507 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/110 2024-04-01T01:08:45.507 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:45.509 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:45.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:45.518 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-04-01T01:08:45.933 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:45.939 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:46.102 INFO:teuthology.orchestra.run.smithi064.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:08:46.106 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/111 2024-04-01T01:08:46.106 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/111 2024-04-01T01:08:46.106 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:08:46.108 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:46.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:08:46.117 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-04-01T01:08:46.566 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:08:46.570 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:08:46.570 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:08:55.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:08:55.127 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:09:24.444 INFO:teuthology.orchestra.run.smithi064.stdout:generic/112 38s 2024-04-01T01:09:24.448 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/112 2024-04-01T01:09:24.449 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:09:24.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:09:24.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:09:24.460 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-04-01T01:09:24.880 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:09:24.885 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:09:24.885 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:09:25.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:09:25.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:09:55.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:09:55.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:10:04.973 INFO:teuthology.orchestra.run.smithi064.stdout:generic/113 40s 2024-04-01T01:10:04.978 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/113 2024-04-01T01:10:04.978 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:05.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:05.001 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-04-01T01:10:05.466 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:05.471 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:05.471 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:05.619 INFO:teuthology.orchestra.run.smithi064.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-04-01T01:10:05.623 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/114 2024-04-01T01:10:05.623 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/114 2024-04-01T01:10:05.623 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:05.625 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:05.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:05.634 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-04-01T01:10:05.806 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:05.810 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:05.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:05.962 INFO:teuthology.orchestra.run.smithi064.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/115 2024-04-01T01:10:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/115 2024-04-01T01:10:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:05.967 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:05.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:05.976 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-04-01T01:10:06.398 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:06.404 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:06.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:06.567 INFO:teuthology.orchestra.run.smithi064.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:06.571 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/116 2024-04-01T01:10:06.571 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/116 2024-04-01T01:10:06.571 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:06.573 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:06.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:06.582 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-04-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:07.007 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:07.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:07.124 INFO:teuthology.orchestra.run.smithi064.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:10:07.127 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/117 2024-04-01T01:10:07.127 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/117 2024-04-01T01:10:07.127 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:07.129 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:07.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:07.138 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-04-01T01:10:07.313 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:07.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:07.472 INFO:teuthology.orchestra.run.smithi064.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:07.476 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/118 2024-04-01T01:10:07.476 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/118 2024-04-01T01:10:07.476 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:07.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:07.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:07.486 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-04-01T01:10:07.908 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:07.912 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:07.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.069 INFO:teuthology.orchestra.run.smithi064.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:08.073 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/119 2024-04-01T01:10:08.073 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/119 2024-04-01T01:10:08.073 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:08.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:08.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-04-01T01:10:08.505 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:08.511 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:08.511 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.622 INFO:teuthology.orchestra.run.smithi064.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:10:08.626 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/120 2024-04-01T01:10:08.626 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/120 2024-04-01T01:10:08.626 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:08.628 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:08.637 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-04-01T01:10:08.810 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:08.815 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:08.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.975 INFO:teuthology.orchestra.run.smithi064.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/121 2024-04-01T01:10:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/121 2024-04-01T01:10:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:08.981 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:08.989 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:08.990 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-04-01T01:10:09.419 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:09.424 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:09.424 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:10:09.593 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/122 2024-04-01T01:10:09.593 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/122 2024-04-01T01:10:09.593 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:09.595 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:09.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:09.604 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-04-01T01:10:10.044 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:10.050 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:10.051 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:10.590 INFO:teuthology.orchestra.run.smithi064.stdout:generic/123 0s 2024-04-01T01:10:10.595 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/123 2024-04-01T01:10:10.595 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:10.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:10.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:10.606 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-04-01T01:10:11.032 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:11.035 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:11.035 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:11.156 INFO:teuthology.orchestra.run.smithi064.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:10:11.159 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/124 2024-04-01T01:10:11.159 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/124 2024-04-01T01:10:11.159 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:10:11.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:11.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:10:11.170 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-04-01T01:10:11.362 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:10:11.365 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:10:11.366 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:10:25.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:10:25.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:10:55.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:10:55.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:11:12.721 INFO:teuthology.orchestra.run.smithi064.stdout:generic/125 61s 2024-04-01T01:11:12.728 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/125 2024-04-01T01:11:12.728 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:11:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:11:12.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:11:12.745 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-04-01T01:11:13.205 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:11:13.209 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:11:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:11:13.692 INFO:teuthology.orchestra.run.smithi064.stdout:generic/126 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/126.out.bad) 2024-04-01T01:11:13.693 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/126.out 2024-04-01 00:14:57.000908382 +0000 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/126.out.bad 2024-04-01 01:11:13.690892241 +0000 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,7 +1,7 @@ 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 126 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2024-04-01T01:11:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2024-04-01T01:11:13.695 INFO:teuthology.orchestra.run.smithi064.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2024-04-01T01:11:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/126 2024-04-01T01:11:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/126 2024-04-01T01:11:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:11:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:11:13.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:11:13.707 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:11:13.707 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:11:13.707 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-04-01T01:11:14.146 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:11:14.150 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:11:14.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:11:25.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:11:25.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:11:55.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:11:55.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:11:55.890 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T01:11:55.891+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 191609) UID: 0 2024-04-01T01:11:55.924 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T01:11:55.891+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 191609) UID: 0 2024-04-01T01:11:55.924 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T01:11:55.891+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 191609) UID: 0 2024-04-01T01:11:55.925 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T01:11:55.891+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 191609) UID: 0 2024-04-01T01:14:35.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:14:35.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:15:05.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:15:05.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:15:35.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:15:35.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:16:05.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:16:05.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:16:35.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:16:35.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:17:05.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:17:05.871 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:17:35.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:17:35.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:18:06.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:18:06.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:18:36.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:18:36.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:18:36.354 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T01:18:36.354+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46612) UID: 0 2024-04-01T01:18:36.354 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T01:18:36.354+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46612) UID: 0 2024-04-01T01:18:36.355 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T01:18:36.354+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46612) UID: 0 2024-04-01T01:18:36.355 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T01:18:36.354+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46612) UID: 0 2024-04-01T01:21:20.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:21:20.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:21:50.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:21:50.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:22:20.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:22:20.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:22:50.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:22:50.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:23:20.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:23:20.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:23:50.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:23:50.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:24:20.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:24:21.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:24:43.928 INFO:teuthology.orchestra.run.smithi064.stdout:generic/127 809s 2024-04-01T01:24:43.933 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/127 2024-04-01T01:24:43.933 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:43.935 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:43.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:43.944 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-04-01T01:24:44.378 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:44.383 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:44.383 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:44.512 INFO:teuthology.orchestra.run.smithi064.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:24:44.515 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/128 2024-04-01T01:24:44.515 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/128 2024-04-01T01:24:44.515 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:44.517 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:44.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:44.526 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-04-01T01:24:44.702 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:44.705 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:44.706 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:44.819 INFO:teuthology.orchestra.run.smithi064.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:24:44.823 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/129 2024-04-01T01:24:44.823 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/129 2024-04-01T01:24:44.823 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:44.825 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:44.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:44.833 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-04-01T01:24:45.008 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:45.012 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:45.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:45.130 INFO:teuthology.orchestra.run.smithi064.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:24:45.134 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/130 2024-04-01T01:24:45.134 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/130 2024-04-01T01:24:45.134 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:45.136 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:45.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:45.145 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-04-01T01:24:45.327 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:45.331 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:45.331 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:46.631 INFO:teuthology.orchestra.run.smithi064.stdout:generic/131 1s 2024-04-01T01:24:46.636 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/131 2024-04-01T01:24:46.636 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:46.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:46.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:46.647 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-04-01T01:24:47.079 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:47.083 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:47.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:47.202 INFO:teuthology.orchestra.run.smithi064.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:24:47.206 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/132 2024-04-01T01:24:47.206 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/132 2024-04-01T01:24:47.206 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:24:47.208 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:47.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:24:47.216 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-04-01T01:24:47.389 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:24:47.393 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:24:47.393 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:24:51.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:24:51.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:25:21.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:25:21.431 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:25:51.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:25:51.553 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:26:21.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:26:21.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:26:51.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:26:51.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:26:51.845 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T01:26:51.845+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46841) UID: 0 2024-04-01T01:26:51.845 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T01:26:51.845+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46841) UID: 0 2024-04-01T01:26:51.846 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T01:26:51.845+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46841) UID: 0 2024-04-01T01:26:51.846 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T01:26:51.845+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46841) UID: 0 2024-04-01T01:28:38.972 INFO:teuthology.orchestra.run.smithi064.stdout:generic/133 231s 2024-04-01T01:28:38.976 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/133 2024-04-01T01:28:38.976 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:38.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:38.988 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-04-01T01:28:39.405 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:39.409 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:39.409 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:39.581 INFO:teuthology.orchestra.run.smithi064.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:39.584 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/134 2024-04-01T01:28:39.584 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/134 2024-04-01T01:28:39.584 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:39.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:39.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:39.595 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-04-01T01:28:40.037 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:40.041 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:40.041 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:40.324 INFO:teuthology.orchestra.run.smithi064.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:40.328 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/135 2024-04-01T01:28:40.328 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/135 2024-04-01T01:28:40.328 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:40.330 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:40.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:40.338 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-04-01T01:28:40.515 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:40.519 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:40.519 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:40.686 INFO:teuthology.orchestra.run.smithi064.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:40.690 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/136 2024-04-01T01:28:40.690 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/136 2024-04-01T01:28:40.690 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:40.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:40.700 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-04-01T01:28:41.103 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:41.107 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:41.107 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:41.271 INFO:teuthology.orchestra.run.smithi064.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:41.274 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/137 2024-04-01T01:28:41.274 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/137 2024-04-01T01:28:41.275 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:41.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:41.284 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-04-01T01:28:41.703 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:41.707 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:41.869 INFO:teuthology.orchestra.run.smithi064.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:41.872 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/138 2024-04-01T01:28:41.873 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/138 2024-04-01T01:28:41.873 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:41.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:41.883 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-04-01T01:28:42.290 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:42.295 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:42.464 INFO:teuthology.orchestra.run.smithi064.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:42.467 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/139 2024-04-01T01:28:42.467 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/139 2024-04-01T01:28:42.467 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:42.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:42.478 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-04-01T01:28:42.893 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:42.900 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.058 INFO:teuthology.orchestra.run.smithi064.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:43.062 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/140 2024-04-01T01:28:43.062 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/140 2024-04-01T01:28:43.062 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:43.064 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:43.072 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-04-01T01:28:43.469 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:43.472 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:43.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.591 INFO:teuthology.orchestra.run.smithi064.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:43.594 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/141 2024-04-01T01:28:43.594 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/141 2024-04-01T01:28:43.594 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:43.596 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:43.604 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-04-01T01:28:43.777 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:43.780 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.942 INFO:teuthology.orchestra.run.smithi064.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:43.945 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/142 2024-04-01T01:28:43.945 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/142 2024-04-01T01:28:43.945 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:43.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:43.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:43.955 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-04-01T01:28:44.377 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:44.382 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:44.382 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:44.545 INFO:teuthology.orchestra.run.smithi064.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:44.548 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/143 2024-04-01T01:28:44.548 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/143 2024-04-01T01:28:44.548 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:44.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:44.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:44.558 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-04-01T01:28:44.957 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:44.962 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:45.126 INFO:teuthology.orchestra.run.smithi064.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:45.129 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/144 2024-04-01T01:28:45.129 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/144 2024-04-01T01:28:45.129 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:45.131 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:45.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:45.140 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-04-01T01:28:45.559 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:45.562 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:45.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:45.727 INFO:teuthology.orchestra.run.smithi064.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:45.730 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/145 2024-04-01T01:28:45.730 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/145 2024-04-01T01:28:45.730 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:45.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:45.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:45.741 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-04-01T01:28:46.138 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:46.359 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:46.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:46.360 INFO:teuthology.orchestra.run.smithi064.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:46.360 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/146 2024-04-01T01:28:46.360 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/146 2024-04-01T01:28:46.360 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:46.360 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:46.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:46.360 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-04-01T01:28:46.833 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:46.837 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:46.837 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:46.997 INFO:teuthology.orchestra.run.smithi064.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:47.000 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/147 2024-04-01T01:28:47.000 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/147 2024-04-01T01:28:47.000 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:47.002 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:47.055 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:47.055 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-04-01T01:28:47.452 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:47.456 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:47.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:47.623 INFO:teuthology.orchestra.run.smithi064.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:47.627 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/148 2024-04-01T01:28:47.627 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/148 2024-04-01T01:28:47.627 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:47.629 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:47.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:47.638 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-04-01T01:28:48.045 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:48.049 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:48.049 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:48.218 INFO:teuthology.orchestra.run.smithi064.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:48.222 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/149 2024-04-01T01:28:48.222 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/149 2024-04-01T01:28:48.222 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:48.224 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:48.232 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:48.232 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-04-01T01:28:48.633 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:48.637 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:48.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:48.820 INFO:teuthology.orchestra.run.smithi064.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:48.823 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/150 2024-04-01T01:28:48.823 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/150 2024-04-01T01:28:48.823 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:48.826 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:48.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:48.834 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-04-01T01:28:49.230 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:49.236 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:49.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:49.404 INFO:teuthology.orchestra.run.smithi064.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:49.408 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/151 2024-04-01T01:28:49.408 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/151 2024-04-01T01:28:49.408 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:49.410 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:49.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:49.418 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-04-01T01:28:49.802 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:49.807 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:49.807 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:49.974 INFO:teuthology.orchestra.run.smithi064.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:49.977 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/152 2024-04-01T01:28:49.977 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/152 2024-04-01T01:28:49.977 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:49.979 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:49.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:49.987 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-04-01T01:28:50.387 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:50.391 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:50.391 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:50.559 INFO:teuthology.orchestra.run.smithi064.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:50.562 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/153 2024-04-01T01:28:50.563 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/153 2024-04-01T01:28:50.563 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:50.565 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:50.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:50.608 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-04-01T01:28:50.969 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:50.974 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:50.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:51.148 INFO:teuthology.orchestra.run.smithi064.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:51.151 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/154 2024-04-01T01:28:51.151 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/154 2024-04-01T01:28:51.151 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:51.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:51.172 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-04-01T01:28:51.581 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:51.585 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:51.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:51.772 INFO:teuthology.orchestra.run.smithi064.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:51.776 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/155 2024-04-01T01:28:51.776 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/155 2024-04-01T01:28:51.776 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:51.778 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:51.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:51.786 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-04-01T01:28:52.193 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:52.198 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:52.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:52.490 INFO:teuthology.orchestra.run.smithi064.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:52.494 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/156 2024-04-01T01:28:52.494 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/156 2024-04-01T01:28:52.494 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:52.496 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:52.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:52.504 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-04-01T01:28:52.923 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:52.927 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:52.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:53.097 INFO:teuthology.orchestra.run.smithi064.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:53.100 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/157 2024-04-01T01:28:53.100 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/157 2024-04-01T01:28:53.100 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:53.102 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:53.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:53.110 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-04-01T01:28:53.632 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:53.636 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:53.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:53.803 INFO:teuthology.orchestra.run.smithi064.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:28:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/158 2024-04-01T01:28:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/158 2024-04-01T01:28:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:53.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:53.817 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:53.817 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-04-01T01:28:54.213 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:54.217 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:54.399 INFO:teuthology.orchestra.run.smithi064.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-04-01T01:28:54.403 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/159 2024-04-01T01:28:54.403 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/159 2024-04-01T01:28:54.403 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:54.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:54.413 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-04-01T01:28:54.590 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:54.594 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:54.743 INFO:teuthology.orchestra.run.smithi064.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-04-01T01:28:54.746 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/160 2024-04-01T01:28:54.746 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/160 2024-04-01T01:28:54.747 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:54.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:54.757 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-04-01T01:28:54.923 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:54.927 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.056 INFO:teuthology.orchestra.run.smithi064.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:55.059 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/161 2024-04-01T01:28:55.059 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/161 2024-04-01T01:28:55.059 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:55.061 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:55.070 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-04-01T01:28:55.250 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:55.254 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:55.254 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.381 INFO:teuthology.orchestra.run.smithi064.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:55.384 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/162 2024-04-01T01:28:55.384 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/162 2024-04-01T01:28:55.384 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:55.395 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-04-01T01:28:55.575 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:55.578 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:55.579 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.706 INFO:teuthology.orchestra.run.smithi064.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:55.709 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/163 2024-04-01T01:28:55.709 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/163 2024-04-01T01:28:55.709 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:55.712 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:55.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:55.734 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-04-01T01:28:55.912 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:55.916 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.032 INFO:teuthology.orchestra.run.smithi064.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:56.036 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/164 2024-04-01T01:28:56.036 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/164 2024-04-01T01:28:56.036 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:56.038 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:56.046 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-04-01T01:28:56.219 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:56.222 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:56.223 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/165 2024-04-01T01:28:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/165 2024-04-01T01:28:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:56.350 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:56.358 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-04-01T01:28:56.531 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:56.535 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:56.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.656 INFO:teuthology.orchestra.run.smithi064.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:56.660 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/166 2024-04-01T01:28:56.660 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/166 2024-04-01T01:28:56.660 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:56.662 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:56.670 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-04-01T01:28:56.846 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:56.850 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:56.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.972 INFO:teuthology.orchestra.run.smithi064.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:56.976 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/167 2024-04-01T01:28:56.976 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/167 2024-04-01T01:28:56.976 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:56.978 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:56.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:56.986 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-04-01T01:28:57.158 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:57.161 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:57.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.273 INFO:teuthology.orchestra.run.smithi064.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:57.277 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/168 2024-04-01T01:28:57.277 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/168 2024-04-01T01:28:57.277 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:57.279 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:57.287 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-04-01T01:28:57.462 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:57.465 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:57.466 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.578 INFO:teuthology.orchestra.run.smithi064.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:57.581 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/169 2024-04-01T01:28:57.581 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/169 2024-04-01T01:28:57.581 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:57.583 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:57.592 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-04-01T01:28:57.765 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:57.769 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:57.769 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.890 INFO:teuthology.orchestra.run.smithi064.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:57.893 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/170 2024-04-01T01:28:57.893 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/170 2024-04-01T01:28:57.894 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:57.896 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:57.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:57.904 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-04-01T01:28:58.076 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:58.080 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.213 INFO:teuthology.orchestra.run.smithi064.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:58.217 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/171 2024-04-01T01:28:58.217 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/171 2024-04-01T01:28:58.217 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:58.237 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-04-01T01:28:58.408 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:58.412 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.538 INFO:teuthology.orchestra.run.smithi064.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:58.541 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/172 2024-04-01T01:28:58.542 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/172 2024-04-01T01:28:58.542 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:58.552 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-04-01T01:28:58.729 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:58.733 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.854 INFO:teuthology.orchestra.run.smithi064.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:58.858 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/173 2024-04-01T01:28:58.858 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/173 2024-04-01T01:28:58.858 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:58.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:58.868 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-04-01T01:28:59.046 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:59.049 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:59.172 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/174 2024-04-01T01:28:59.172 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/174 2024-04-01T01:28:59.172 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:59.182 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-04-01T01:28:59.355 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:59.359 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:59.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.484 INFO:teuthology.orchestra.run.smithi064.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/175 2024-04-01T01:28:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/175 2024-04-01T01:28:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:59.489 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:59.497 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-04-01T01:28:59.672 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:59.675 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:59.676 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.795 INFO:teuthology.orchestra.run.smithi064.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:28:59.798 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/176 2024-04-01T01:28:59.798 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/176 2024-04-01T01:28:59.798 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:28:59.800 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:28:59.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:28:59.823 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-04-01T01:28:59.994 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:28:59.998 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:28:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:00.134 INFO:teuthology.orchestra.run.smithi064.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:00.138 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/177 2024-04-01T01:29:00.138 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/177 2024-04-01T01:29:00.138 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:00.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:00.148 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-04-01T01:29:00.330 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:00.334 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:00.493 INFO:teuthology.orchestra.run.smithi064.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:29:00.496 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/178 2024-04-01T01:29:00.496 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/178 2024-04-01T01:29:00.496 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:00.498 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:00.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:00.506 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-04-01T01:29:00.965 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:00.968 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:00.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:01.129 INFO:teuthology.orchestra.run.smithi064.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:29:01.132 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/179 2024-04-01T01:29:01.132 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/179 2024-04-01T01:29:01.132 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:01.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:01.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:01.142 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-04-01T01:29:01.539 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:01.543 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:01.543 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:01.712 INFO:teuthology.orchestra.run.smithi064.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:29:01.715 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/180 2024-04-01T01:29:01.715 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/180 2024-04-01T01:29:01.715 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:01.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:01.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:01.725 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-04-01T01:29:02.128 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:02.133 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:02.133 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:02.297 INFO:teuthology.orchestra.run.smithi064.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:29:02.301 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/181 2024-04-01T01:29:02.301 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/181 2024-04-01T01:29:02.301 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:02.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:02.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:02.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-04-01T01:29:02.781 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:02.785 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:02.943 INFO:teuthology.orchestra.run.smithi064.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:29:02.946 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/182 2024-04-01T01:29:02.946 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/182 2024-04-01T01:29:02.946 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:02.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:02.957 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-04-01T01:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:03.470 INFO:teuthology.orchestra.run.smithi064.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:03.473 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/183 2024-04-01T01:29:03.473 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/183 2024-04-01T01:29:03.473 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:03.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:03.483 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-04-01T01:29:03.657 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:03.660 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:03.823 INFO:teuthology.orchestra.run.smithi064.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/184.out.bad) 2024-04-01T01:29:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/184.out 2024-04-01 00:14:57.004908302 +0000 2024-04-01T01:29:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/184.out.bad 2024-04-01 01:29:03.822109561 +0000 2024-04-01T01:29:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1 +1,2 @@ 2024-04-01T01:29:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 184 2024-04-01T01:29:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: +/tmp/tmp.ALJJcUYdNQxfstests-dev/tests/generic/184: line 23: /tmp/tmp.vihoW6DmoUtest/null: Permission denied 2024-04-01T01:29:03.827 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/184 2024-04-01T01:29:03.827 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/184 2024-04-01T01:29:03.827 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:29:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:03.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:29:03.838 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:29:03.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:29:03.839 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-04-01T01:29:04.013 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:04.016 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.131 INFO:teuthology.orchestra.run.smithi064.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:04.135 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/185 2024-04-01T01:29:04.135 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/185 2024-04-01T01:29:04.135 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:04.145 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-04-01T01:29:04.316 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:04.320 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:04.320 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.431 INFO:teuthology.orchestra.run.smithi064.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:04.434 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/186 2024-04-01T01:29:04.434 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/186 2024-04-01T01:29:04.434 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:04.444 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-04-01T01:29:04.618 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:04.623 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:04.623 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.755 INFO:teuthology.orchestra.run.smithi064.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/187 2024-04-01T01:29:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/187 2024-04-01T01:29:04.759 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:04.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:04.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:04.769 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-04-01T01:29:04.958 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:04.962 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:04.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.087 INFO:teuthology.orchestra.run.smithi064.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:05.090 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/188 2024-04-01T01:29:05.090 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/188 2024-04-01T01:29:05.090 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:05.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.100 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:05.101 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-04-01T01:29:05.282 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:05.286 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:05.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.431 INFO:teuthology.orchestra.run.smithi064.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:05.435 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/189 2024-04-01T01:29:05.435 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/189 2024-04-01T01:29:05.435 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:05.437 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:05.445 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-04-01T01:29:05.625 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:05.628 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:05.629 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.755 INFO:teuthology.orchestra.run.smithi064.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:05.759 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/190 2024-04-01T01:29:05.759 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/190 2024-04-01T01:29:05.759 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:05.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:05.769 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:05.769 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-04-01T01:29:05.939 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:05.942 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:06.056 INFO:teuthology.orchestra.run.smithi064.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:06.060 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/191 2024-04-01T01:29:06.060 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/191 2024-04-01T01:29:06.060 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:06.062 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:06.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:06.070 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-04-01T01:29:06.245 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:06.249 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:06.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:06.374 INFO:teuthology.orchestra.run.smithi064.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2024-04-01T01:29:06.378 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/192 2024-04-01T01:29:06.378 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/192 2024-04-01T01:29:06.378 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:06.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:06.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:06.402 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-04-01T01:29:06.579 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:06.583 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:06.583 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:07.518 INFO:teuthology.orchestra.run.smithi064.stdout:generic/193 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/193.out.bad) 2024-04-01T01:29:07.520 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/193.out 2024-04-01 00:14:57.005908282 +0000 2024-04-01T01:29:07.520 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/193.out.bad 2024-04-01 01:29:07.517037846 +0000 2024-04-01T01:29:07.520 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -24,7 +24,6 @@ 2024-04-01T01:29:07.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:07.520 INFO:teuthology.orchestra.run.smithi064.stdout: user: chmod a+r on qa_user owned file (should succeed) 2024-04-01T01:29:07.521 INFO:teuthology.orchestra.run.smithi064.stdout: user: chmod a+r on root owned file (should fail) 2024-04-01T01:29:07.521 INFO:teuthology.orchestra.run.smithi064.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2024-04-01T01:29:07.521 INFO:teuthology.orchestra.run.smithi064.stdout: check that the sgid bit is cleared 2024-04-01T01:29:07.521 INFO:teuthology.orchestra.run.smithi064.stdout: -rw-rw-rw- 2024-04-01T01:29:07.521 INFO:teuthology.orchestra.run.smithi064.stdout: check that suid bit is not cleared 2024-04-01T01:29:07.523 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/193 2024-04-01T01:29:07.523 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/193 2024-04-01T01:29:07.523 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:29:07.525 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:07.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:29:07.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:29:07.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:29:07.533 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-04-01T01:29:07.970 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:07.974 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:07.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.087 INFO:teuthology.orchestra.run.smithi064.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:08.091 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/194 2024-04-01T01:29:08.091 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/194 2024-04-01T01:29:08.091 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:08.093 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:08.101 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-04-01T01:29:08.285 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:08.289 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:08.289 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.412 INFO:teuthology.orchestra.run.smithi064.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:08.416 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/195 2024-04-01T01:29:08.416 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/195 2024-04-01T01:29:08.416 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:08.418 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:08.437 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-04-01T01:29:08.609 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:08.613 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:08.613 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.730 INFO:teuthology.orchestra.run.smithi064.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:08.733 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/196 2024-04-01T01:29:08.733 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/196 2024-04-01T01:29:08.734 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:08.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:08.744 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-04-01T01:29:08.917 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:08.920 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:08.921 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:09.036 INFO:teuthology.orchestra.run.smithi064.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:09.039 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/197 2024-04-01T01:29:09.039 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/197 2024-04-01T01:29:09.039 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:09.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:09.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:09.050 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-04-01T01:29:09.231 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:09.235 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:09.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:10.037 INFO:teuthology.orchestra.run.smithi064.stdout:generic/198 0s 2024-04-01T01:29:10.042 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/198 2024-04-01T01:29:10.042 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:10.044 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:10.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:10.053 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-04-01T01:29:10.520 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:10.525 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:10.525 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:10.647 INFO:teuthology.orchestra.run.smithi064.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:10.650 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/199 2024-04-01T01:29:10.650 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/199 2024-04-01T01:29:10.651 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:10.653 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:10.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:10.662 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-04-01T01:29:10.872 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:10.876 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:10.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:10.989 INFO:teuthology.orchestra.run.smithi064.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/200 2024-04-01T01:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/200 2024-04-01T01:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:11.023 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-04-01T01:29:11.209 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.332 INFO:teuthology.orchestra.run.smithi064.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/201 2024-04-01T01:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/201 2024-04-01T01:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:11.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:11.346 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-04-01T01:29:11.528 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.651 INFO:teuthology.orchestra.run.smithi064.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/202 2024-04-01T01:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/202 2024-04-01T01:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:11.657 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:11.665 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-04-01T01:29:11.841 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.962 INFO:teuthology.orchestra.run.smithi064.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/203 2024-04-01T01:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/203 2024-04-01T01:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:11.968 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:11.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:11.976 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-04-01T01:29:12.151 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:12.155 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:12.492 INFO:teuthology.orchestra.run.smithi064.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:12.495 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/204 2024-04-01T01:29:12.495 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/204 2024-04-01T01:29:12.496 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:12.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:12.519 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-04-01T01:29:12.692 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:12.814 INFO:teuthology.orchestra.run.smithi064.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/205 2024-04-01T01:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/205 2024-04-01T01:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:12.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:12.828 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:12.828 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-04-01T01:29:13.006 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:13.135 INFO:teuthology.orchestra.run.smithi064.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/206 2024-04-01T01:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/206 2024-04-01T01:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:13.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:13.148 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-04-01T01:29:13.319 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:13.322 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:33.783 INFO:teuthology.orchestra.run.smithi064.stdout:generic/207 20s 2024-04-01T01:29:33.788 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/207 2024-04-01T01:29:33.788 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:29:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:33.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:29:33.799 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-04-01T01:29:34.222 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:29:34.228 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:29:34.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:29:35.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:29:35.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:30:05.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:30:05.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:30:35.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:30:35.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:31:05.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:31:05.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:31:35.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:31:35.890 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:32:06.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:32:06.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:32:36.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:32:36.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:32:54.465 INFO:teuthology.orchestra.run.smithi064.stdout:generic/208 200s 2024-04-01T01:32:54.470 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/208 2024-04-01T01:32:54.470 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:32:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:32:54.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:32:54.481 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-04-01T01:32:54.924 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:32:54.928 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:32:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:03.413 INFO:teuthology.orchestra.run.smithi064.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/209.out.bad) 2024-04-01T01:33:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/209.out 2024-04-01 00:14:57.006908262 +0000 2024-04-01T01:33:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/209.out.bad 2024-04-01 01:33:03.407447060 +0000 2024-04-01T01:33:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,2 +1,2 @@ 2024-04-01T01:33:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 209 2024-04-01T01:33:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: -test ran for 30 seconds without error 2024-04-01T01:33:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: +reader found old byte at pos 4096 2024-04-01T01:33:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: \ No newline at end of file 2024-04-01T01:33:03.427 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/209 2024-04-01T01:33:03.427 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/209 2024-04-01T01:33:03.427 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:33:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:03.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:33:03.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:33:03.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:33:03.453 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-04-01T01:33:03.839 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:03.843 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:04.076 INFO:teuthology.orchestra.run.smithi064.stdout:generic/210 1s 2024-04-01T01:33:04.081 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/210 2024-04-01T01:33:04.081 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:04.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:04.116 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-04-01T01:33:04.516 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:04.520 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:04.808 INFO:teuthology.orchestra.run.smithi064.stdout:generic/211 0s 2024-04-01T01:33:04.813 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/211 2024-04-01T01:33:04.813 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:04.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:04.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:04.823 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-04-01T01:33:05.003 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:05.006 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:05.212 INFO:teuthology.orchestra.run.smithi064.stdout:generic/212 0s 2024-04-01T01:33:05.218 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/212 2024-04-01T01:33:05.218 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:05.220 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:05.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:05.228 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-04-01T01:33:05.402 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:05.406 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:05.847 INFO:teuthology.orchestra.run.smithi064.stdout:generic/213 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/213.out.bad) 2024-04-01T01:33:05.849 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/213.out 2024-04-01 00:14:57.006908262 +0000 2024-04-01T01:33:05.849 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/213.out.bad 2024-04-01 01:33:05.847399534 +0000 2024-04-01T01:33:05.849 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,4 +1,3 @@ 2024-04-01T01:33:05.849 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 213 2024-04-01T01:33:05.850 INFO:teuthology.orchestra.run.smithi064.stdout: We should get: fallocate: No space left on device 2024-04-01T01:33:05.850 INFO:teuthology.orchestra.run.smithi064.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2024-04-01T01:33:05.850 INFO:teuthology.orchestra.run.smithi064.stdout: -fallocate: No space left on device 2024-04-01T01:33:05.852 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/213 2024-04-01T01:33:05.852 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/213 2024-04-01T01:33:05.852 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:33:05.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:05.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:33:05.867 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:33:05.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:33:05.867 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-04-01T01:33:06.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:33:06.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:33:06.268 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:06.272 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:06.273 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:06.589 INFO:teuthology.orchestra.run.smithi064.stdout:generic/214 0s 2024-04-01T01:33:06.594 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/214 2024-04-01T01:33:06.595 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:06.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:06.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:06.605 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-04-01T01:33:07.050 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:07.055 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:07.056 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.083 INFO:teuthology.orchestra.run.smithi064.stdout:generic/215 3s 2024-04-01T01:33:10.092 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/215 2024-04-01T01:33:10.092 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:10.095 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:10.105 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-04-01T01:33:10.345 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:10.349 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:10.349 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.495 INFO:teuthology.orchestra.run.smithi064.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:10.498 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/216 2024-04-01T01:33:10.499 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/216 2024-04-01T01:33:10.499 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:10.512 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-04-01T01:33:10.749 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:10.755 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:10.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.916 INFO:teuthology.orchestra.run.smithi064.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:10.921 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/217 2024-04-01T01:33:10.921 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/217 2024-04-01T01:33:10.921 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:10.923 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:10.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:10.933 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-04-01T01:33:11.165 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:11.169 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:11.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:11.316 INFO:teuthology.orchestra.run.smithi064.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:11.321 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/218 2024-04-01T01:33:11.321 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/218 2024-04-01T01:33:11.321 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:11.324 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:11.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:11.334 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-04-01T01:33:11.577 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:11.581 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:11.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:11.723 INFO:teuthology.orchestra.run.smithi064.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:11.727 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/219 2024-04-01T01:33:11.727 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/219 2024-04-01T01:33:11.727 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:11.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:11.739 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:11.739 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-04-01T01:33:11.976 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:11.979 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:11.979 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:12.137 INFO:teuthology.orchestra.run.smithi064.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:12.142 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/220 2024-04-01T01:33:12.142 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/220 2024-04-01T01:33:12.142 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:12.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:12.155 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-04-01T01:33:12.392 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:12.396 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:12.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:13.592 INFO:teuthology.orchestra.run.smithi064.stdout:generic/221 1s 2024-04-01T01:33:13.602 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/221 2024-04-01T01:33:13.602 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:13.605 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:13.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:13.616 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-04-01T01:33:13.853 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:13.858 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.015 INFO:teuthology.orchestra.run.smithi064.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:14.019 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/222 2024-04-01T01:33:14.019 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/222 2024-04-01T01:33:14.019 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:14.021 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:14.032 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-04-01T01:33:14.273 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:14.279 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.426 INFO:teuthology.orchestra.run.smithi064.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:14.430 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/223 2024-04-01T01:33:14.430 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/223 2024-04-01T01:33:14.430 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:14.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-04-01T01:33:14.670 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:14.675 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:14.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.832 INFO:teuthology.orchestra.run.smithi064.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:14.837 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/224 2024-04-01T01:33:14.838 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/224 2024-04-01T01:33:14.838 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:14.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:14.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:14.850 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-04-01T01:33:15.085 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:15.090 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:15.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:15.246 INFO:teuthology.orchestra.run.smithi064.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/225 2024-04-01T01:33:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/225 2024-04-01T01:33:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:15.254 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:15.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:15.263 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-04-01T01:33:15.526 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:15.530 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:15.530 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:15.678 INFO:teuthology.orchestra.run.smithi064.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:15.683 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/226 2024-04-01T01:33:15.683 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/226 2024-04-01T01:33:15.683 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:15.687 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:15.696 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:15.697 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-04-01T01:33:15.927 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:15.930 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:15.930 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:16.073 INFO:teuthology.orchestra.run.smithi064.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:16.077 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/227 2024-04-01T01:33:16.077 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/227 2024-04-01T01:33:16.077 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:16.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:16.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:16.090 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-04-01T01:33:16.328 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:16.332 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:16.332 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:16.736 INFO:teuthology.orchestra.run.smithi064.stdout:generic/228 0s 2024-04-01T01:33:16.742 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/228 2024-04-01T01:33:16.742 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:16.760 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:16.761 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-04-01T01:33:17.188 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:17.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:17.345 INFO:teuthology.orchestra.run.smithi064.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:17.349 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/229 2024-04-01T01:33:17.349 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/229 2024-04-01T01:33:17.349 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:17.352 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:17.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:17.361 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-04-01T01:33:17.586 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:17.591 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:17.741 INFO:teuthology.orchestra.run.smithi064.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:17.746 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/230 2024-04-01T01:33:17.746 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/230 2024-04-01T01:33:17.747 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:17.750 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:17.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:17.760 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-04-01T01:33:17.999 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:18.003 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:18.004 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.158 INFO:teuthology.orchestra.run.smithi064.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:18.161 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/231 2024-04-01T01:33:18.161 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/231 2024-04-01T01:33:18.162 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:18.177 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-04-01T01:33:18.398 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:18.408 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:18.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.566 INFO:teuthology.orchestra.run.smithi064.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:18.570 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/232 2024-04-01T01:33:18.570 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/232 2024-04-01T01:33:18.570 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:18.583 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-04-01T01:33:18.821 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:18.826 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:18.826 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.979 INFO:teuthology.orchestra.run.smithi064.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:18.984 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/233 2024-04-01T01:33:18.984 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/233 2024-04-01T01:33:18.984 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:18.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:18.998 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-04-01T01:33:19.233 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:19.238 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:19.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:19.393 INFO:teuthology.orchestra.run.smithi064.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:19.398 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/234 2024-04-01T01:33:19.398 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/234 2024-04-01T01:33:19.398 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:19.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:19.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:19.415 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-04-01T01:33:19.648 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:19.652 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:19.652 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:19.795 INFO:teuthology.orchestra.run.smithi064.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:19.799 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/235 2024-04-01T01:33:19.799 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/235 2024-04-01T01:33:19.799 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:19.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:19.824 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-04-01T01:33:20.055 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:20.061 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:20.061 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:21.327 INFO:teuthology.orchestra.run.smithi064.stdout:generic/236 1s 2024-04-01T01:33:21.333 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/236 2024-04-01T01:33:21.333 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:21.336 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:21.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:21.347 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-04-01T01:33:21.793 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:21.797 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:21.797 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.005 INFO:teuthology.orchestra.run.smithi064.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T01:33:22.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/237 2024-04-01T01:33:22.009 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/237 2024-04-01T01:33:22.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:22.013 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:22.020 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-04-01T01:33:22.253 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:22.258 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:22.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.405 INFO:teuthology.orchestra.run.smithi064.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:33:22.409 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/238 2024-04-01T01:33:22.409 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/238 2024-04-01T01:33:22.409 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:22.411 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:22.430 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-04-01T01:33:22.676 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:22.681 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.870 INFO:teuthology.orchestra.run.smithi064.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-04-01T01:33:22.874 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/239 2024-04-01T01:33:22.875 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/239 2024-04-01T01:33:22.878 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:22.878 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:22.890 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:22.891 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-04-01T01:33:23.135 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:23.139 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:23.140 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:23.316 INFO:teuthology.orchestra.run.smithi064.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-04-01T01:33:23.319 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/240 2024-04-01T01:33:23.319 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/240 2024-04-01T01:33:23.320 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:33:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:23.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:33:23.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-04-01T01:33:23.554 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:33:23.558 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:33:23.558 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:33:36.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:33:36.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:34:06.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:34:06.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:34:37.003 INFO:teuthology.orchestra.run.smithi064.stdout:generic/241 73s 2024-04-01T01:34:37.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/241 2024-04-01T01:34:37.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:37.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:37.026 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-04-01T01:34:37.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:34:37.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:34:37.642 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:37.646 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:37.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:37.801 INFO:teuthology.orchestra.run.smithi064.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:34:37.806 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/242 2024-04-01T01:34:37.806 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/242 2024-04-01T01:34:37.806 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:37.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:37.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:37.818 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-04-01T01:34:38.062 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:38.066 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:38.215 INFO:teuthology.orchestra.run.smithi064.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:34:38.219 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/243 2024-04-01T01:34:38.220 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/243 2024-04-01T01:34:38.220 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:38.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:38.232 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:38.233 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-04-01T01:34:38.464 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:38.468 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:38.468 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:38.637 INFO:teuthology.orchestra.run.smithi064.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T01:34:38.641 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/244 2024-04-01T01:34:38.641 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/244 2024-04-01T01:34:38.641 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:38.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:38.653 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-04-01T01:34:38.913 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:38.918 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:38.918 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:39.248 INFO:teuthology.orchestra.run.smithi064.stdout:generic/245 1s 2024-04-01T01:34:39.255 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/245 2024-04-01T01:34:39.255 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:39.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:39.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:39.270 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-04-01T01:34:39.928 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:39.942 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:39.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:40.197 INFO:teuthology.orchestra.run.smithi064.stdout:generic/246 1s 2024-04-01T01:34:40.203 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/246 2024-04-01T01:34:40.203 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:34:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:34:40.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:34:40.215 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-04-01T01:34:40.734 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:34:40.740 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:34:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:07.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:35:07.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:35:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:generic/247 46s 2024-04-01T01:35:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/247 2024-04-01T01:35:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:26.633 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:26.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:26.642 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-04-01T01:35:26.886 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:26.890 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:26.890 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:27.154 INFO:teuthology.orchestra.run.smithi064.stdout:generic/248 1s 2024-04-01T01:35:27.160 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/248 2024-04-01T01:35:27.161 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:27.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:27.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:27.174 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-04-01T01:35:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:29.552 INFO:teuthology.orchestra.run.smithi064.stdout:generic/249 2s 2024-04-01T01:35:29.559 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/249 2024-04-01T01:35:29.559 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:29.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:29.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:29.575 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-04-01T01:35:30.086 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:30.090 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:30.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:30.292 INFO:teuthology.orchestra.run.smithi064.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:30.296 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/250 2024-04-01T01:35:30.297 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/250 2024-04-01T01:35:30.297 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:30.300 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:30.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:30.310 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-04-01T01:35:30.544 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:30.552 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:30.552 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:30.739 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/251 2024-04-01T01:35:30.739 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/251 2024-04-01T01:35:30.740 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:30.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:30.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:30.752 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-04-01T01:35:30.986 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:30.993 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:30.993 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:31.164 INFO:teuthology.orchestra.run.smithi064.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:31.169 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/252 2024-04-01T01:35:31.169 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/252 2024-04-01T01:35:31.169 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:31.172 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:31.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:31.183 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-04-01T01:35:31.430 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:31.435 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:31.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:31.593 INFO:teuthology.orchestra.run.smithi064.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:31.597 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/253 2024-04-01T01:35:31.597 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/253 2024-04-01T01:35:31.597 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:31.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:31.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:31.610 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-04-01T01:35:31.841 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:31.845 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:31.846 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:32.005 INFO:teuthology.orchestra.run.smithi064.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:32.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/254 2024-04-01T01:35:32.009 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/254 2024-04-01T01:35:32.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:32.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:32.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:32.022 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-04-01T01:35:32.260 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:32.266 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:33.623 INFO:teuthology.orchestra.run.smithi064.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T01:35:33.627 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/255 2024-04-01T01:35:33.628 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/255 2024-04-01T01:35:33.628 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:33.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:33.646 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-04-01T01:35:34.111 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:34.115 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:34.116 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:35.402 INFO:teuthology.orchestra.run.smithi064.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:35.407 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/256 2024-04-01T01:35:35.407 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/256 2024-04-01T01:35:35.407 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:35.410 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:35.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:35.419 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-04-01T01:35:35.662 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:35.666 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:35.667 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:37.353 INFO:teuthology.orchestra.run.smithi064.stdout:generic/257 2s 2024-04-01T01:35:37.360 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/257 2024-04-01T01:35:37.360 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:37.363 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:37.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:37.373 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-04-01T01:35:37.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:35:37.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:35:37.820 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:37.828 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:37.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.027 INFO:teuthology.orchestra.run.smithi064.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/258.out.bad) 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/258.out 2024-04-01 00:14:57.009908202 +0000 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/258.out.bad 2024-04-01 01:35:38.026442397 +0000 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,5 +1,6 @@ 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 258 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: Creating file with timestamp of Jan 1, 1960 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: Testing for negative seconds since epoch 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: -Remounting to flush cache 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: -Testing for negative seconds since epoch 2024-04-01T01:35:38.029 INFO:teuthology.orchestra.run.smithi064.stdout: +Timestamp wrapped: 3979351756 2024-04-01T01:35:38.030 INFO:teuthology.orchestra.run.smithi064.stdout: +Timestamp wrapped 2024-04-01T01:35:38.030 INFO:teuthology.orchestra.run.smithi064.stdout: +(see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/258.full for details) 2024-04-01T01:35:38.032 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/258 2024-04-01T01:35:38.032 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/258 2024-04-01T01:35:38.032 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:35:38.035 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:35:38.044 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:35:38.044 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:35:38.044 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-04-01T01:35:38.281 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:38.287 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:38.287 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.440 INFO:teuthology.orchestra.run.smithi064.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/259 2024-04-01T01:35:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/259 2024-04-01T01:35:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:38.449 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:38.461 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-04-01T01:35:38.692 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:38.697 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.848 INFO:teuthology.orchestra.run.smithi064.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:38.852 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/260 2024-04-01T01:35:38.852 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/260 2024-04-01T01:35:38.852 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:38.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:38.865 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-04-01T01:35:39.093 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:39.098 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:39.098 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:39.246 INFO:teuthology.orchestra.run.smithi064.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:39.251 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/261 2024-04-01T01:35:39.251 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/261 2024-04-01T01:35:39.251 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:39.254 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:39.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:39.264 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-04-01T01:35:39.486 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:39.491 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:39.491 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:39.639 INFO:teuthology.orchestra.run.smithi064.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:35:39.644 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/262 2024-04-01T01:35:39.644 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/262 2024-04-01T01:35:39.644 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:35:39.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:35:39.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:35:39.669 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-04-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:35:39.905 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:35:39.905 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:36:07.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:36:07.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:36:37.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:36:37.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:37:04.344 INFO:teuthology.orchestra.run.smithi064.stdout:generic/263 85s 2024-04-01T01:37:04.380 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/263 2024-04-01T01:37:04.380 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:04.382 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:04.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:04.391 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-04-01T01:37:04.843 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:04.847 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.018 INFO:teuthology.orchestra.run.smithi064.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:05.021 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/264 2024-04-01T01:37:05.021 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/264 2024-04-01T01:37:05.021 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:05.031 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-04-01T01:37:05.206 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:05.209 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:05.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.365 INFO:teuthology.orchestra.run.smithi064.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:05.369 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/265 2024-04-01T01:37:05.369 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/265 2024-04-01T01:37:05.369 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:05.371 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:05.379 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-04-01T01:37:05.559 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:05.562 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:05.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.709 INFO:teuthology.orchestra.run.smithi064.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:05.712 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/266 2024-04-01T01:37:05.712 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/266 2024-04-01T01:37:05.712 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:05.714 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:05.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:05.723 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-04-01T01:37:05.893 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:05.898 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.043 INFO:teuthology.orchestra.run.smithi064.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:06.046 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/267 2024-04-01T01:37:06.046 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/267 2024-04-01T01:37:06.047 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:06.049 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:06.057 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-04-01T01:37:06.231 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:06.235 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:06.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.373 INFO:teuthology.orchestra.run.smithi064.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:06.377 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/268 2024-04-01T01:37:06.377 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/268 2024-04-01T01:37:06.377 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:06.379 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:06.387 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-04-01T01:37:06.559 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:06.562 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:06.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.674 INFO:teuthology.orchestra.run.smithi064.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:06.678 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/269 2024-04-01T01:37:06.678 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/269 2024-04-01T01:37:06.678 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:06.680 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:06.688 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-04-01T01:37:06.864 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:06.868 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:06.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:06.996 INFO:teuthology.orchestra.run.smithi064.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T01:37:06.999 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/270 2024-04-01T01:37:06.999 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/270 2024-04-01T01:37:06.999 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:07.001 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:07.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-04-01T01:37:07.183 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:07.186 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:07.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.316 INFO:teuthology.orchestra.run.smithi064.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:07.319 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/271 2024-04-01T01:37:07.319 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/271 2024-04-01T01:37:07.319 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:07.321 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:07.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-04-01T01:37:07.503 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:07.506 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:07.507 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.631 INFO:teuthology.orchestra.run.smithi064.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:07.634 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/272 2024-04-01T01:37:07.634 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/272 2024-04-01T01:37:07.634 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:07.645 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-04-01T01:37:07.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:37:07.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:37:07.814 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:07.818 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:07.819 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:07.949 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/273 2024-04-01T01:37:07.949 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/273 2024-04-01T01:37:07.949 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:07.951 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:07.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:07.960 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-04-01T01:37:08.175 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:08.179 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.292 INFO:teuthology.orchestra.run.smithi064.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:08.296 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/274 2024-04-01T01:37:08.296 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/274 2024-04-01T01:37:08.296 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:08.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:08.306 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-04-01T01:37:08.482 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:08.486 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:08.486 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.604 INFO:teuthology.orchestra.run.smithi064.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/275 2024-04-01T01:37:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/275 2024-04-01T01:37:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:08.610 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:08.618 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-04-01T01:37:08.794 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:08.798 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:08.798 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.928 INFO:teuthology.orchestra.run.smithi064.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:08.932 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/276 2024-04-01T01:37:08.932 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/276 2024-04-01T01:37:08.932 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:08.934 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:08.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:08.942 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-04-01T01:37:09.113 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:09.117 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:09.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.240 INFO:teuthology.orchestra.run.smithi064.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:09.243 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/277 2024-04-01T01:37:09.243 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/277 2024-04-01T01:37:09.243 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:09.245 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:09.254 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-04-01T01:37:09.444 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:09.447 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:09.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.579 INFO:teuthology.orchestra.run.smithi064.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:09.582 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/278 2024-04-01T01:37:09.582 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/278 2024-04-01T01:37:09.582 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:09.584 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:09.592 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-04-01T01:37:09.765 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:09.768 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:09.768 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.900 INFO:teuthology.orchestra.run.smithi064.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:09.903 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/279 2024-04-01T01:37:09.904 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/279 2024-04-01T01:37:09.904 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:09.906 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:09.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:09.914 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-04-01T01:37:10.088 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:10.092 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:10.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.245 INFO:teuthology.orchestra.run.smithi064.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/280 2024-04-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/280 2024-04-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:10.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:10.271 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-04-01T01:37:10.447 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:10.451 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:10.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.583 INFO:teuthology.orchestra.run.smithi064.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:10.586 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/281 2024-04-01T01:37:10.587 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/281 2024-04-01T01:37:10.587 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:10.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:10.597 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-04-01T01:37:10.781 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:10.785 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:10.785 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.913 INFO:teuthology.orchestra.run.smithi064.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:10.917 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/282 2024-04-01T01:37:10.917 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/282 2024-04-01T01:37:10.917 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:10.919 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:10.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:10.930 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-04-01T01:37:11.108 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:11.112 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:11.112 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:11.236 INFO:teuthology.orchestra.run.smithi064.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:11.240 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/283 2024-04-01T01:37:11.240 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/283 2024-04-01T01:37:11.240 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:11.242 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:11.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:11.264 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-04-01T01:37:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:11.441 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:11.441 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:11.554 INFO:teuthology.orchestra.run.smithi064.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:11.557 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/284 2024-04-01T01:37:11.557 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/284 2024-04-01T01:37:11.557 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:11.560 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:11.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:11.568 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-04-01T01:37:11.738 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:11.742 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:11.742 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:13.378 INFO:teuthology.orchestra.run.smithi064.stdout:generic/285 2s 2024-04-01T01:37:13.383 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/285 2024-04-01T01:37:13.383 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:13.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:13.393 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-04-01T01:37:13.811 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:13.815 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:13.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:37.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:37:37.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:37:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:generic/286 31s 2024-04-01T01:37:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/286 2024-04-01T01:37:44.083 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:44.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:44.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:44.094 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-04-01T01:37:44.573 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:44.579 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:44.579 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:44.740 INFO:teuthology.orchestra.run.smithi064.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:44.745 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/287 2024-04-01T01:37:44.745 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/287 2024-04-01T01:37:44.745 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:44.748 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:44.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:44.759 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-04-01T01:37:44.951 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:44.954 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.080 INFO:teuthology.orchestra.run.smithi064.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:45.084 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/288 2024-04-01T01:37:45.084 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/288 2024-04-01T01:37:45.084 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:45.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:45.095 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-04-01T01:37:45.279 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:45.282 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:45.283 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.397 INFO:teuthology.orchestra.run.smithi064.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:45.400 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/289 2024-04-01T01:37:45.401 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/289 2024-04-01T01:37:45.401 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:45.403 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:45.411 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-04-01T01:37:45.582 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:45.586 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.699 INFO:teuthology.orchestra.run.smithi064.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:45.703 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/290 2024-04-01T01:37:45.703 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/290 2024-04-01T01:37:45.703 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:45.705 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:45.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:45.714 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-04-01T01:37:45.880 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:45.884 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:45.884 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.007 INFO:teuthology.orchestra.run.smithi064.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:46.010 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/291 2024-04-01T01:37:46.010 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/291 2024-04-01T01:37:46.010 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:46.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:46.020 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-04-01T01:37:46.200 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:46.203 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:46.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.318 INFO:teuthology.orchestra.run.smithi064.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:46.321 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/292 2024-04-01T01:37:46.321 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/292 2024-04-01T01:37:46.321 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:46.323 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:46.332 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-04-01T01:37:46.506 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:46.510 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:46.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.622 INFO:teuthology.orchestra.run.smithi064.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:46.625 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/293 2024-04-01T01:37:46.625 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/293 2024-04-01T01:37:46.625 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:46.628 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:46.636 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-04-01T01:37:46.807 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:46.811 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:46.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.943 INFO:teuthology.orchestra.run.smithi064.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:46.946 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/294 2024-04-01T01:37:46.947 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/294 2024-04-01T01:37:46.947 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:46.949 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:46.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:46.957 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-04-01T01:37:47.125 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:47.129 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:47.129 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:47.295 INFO:teuthology.orchestra.run.smithi064.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:47.299 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/295 2024-04-01T01:37:47.299 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/295 2024-04-01T01:37:47.299 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:47.301 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:47.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:47.310 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-04-01T01:37:47.492 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:47.495 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:47.496 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:47.667 INFO:teuthology.orchestra.run.smithi064.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:47.670 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/296 2024-04-01T01:37:47.670 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/296 2024-04-01T01:37:47.670 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:47.672 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:47.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:47.680 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-04-01T01:37:47.853 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:47.857 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:47.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:48.421 INFO:teuthology.orchestra.run.smithi064.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:48.425 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/297 2024-04-01T01:37:48.425 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/297 2024-04-01T01:37:48.425 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:48.428 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:48.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:48.438 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-04-01T01:37:48.623 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:48.627 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:48.627 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:48.811 INFO:teuthology.orchestra.run.smithi064.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:48.814 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/298 2024-04-01T01:37:48.815 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/298 2024-04-01T01:37:48.815 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:48.817 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:48.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:48.825 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-04-01T01:37:48.996 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:49.000 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:49.000 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.115 INFO:teuthology.orchestra.run.smithi064.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:49.119 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/299 2024-04-01T01:37:49.119 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/299 2024-04-01T01:37:49.119 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:49.120 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:49.129 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-04-01T01:37:49.310 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:49.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.429 INFO:teuthology.orchestra.run.smithi064.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:49.432 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/300 2024-04-01T01:37:49.432 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/300 2024-04-01T01:37:49.432 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:49.434 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:49.443 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-04-01T01:37:49.625 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:49.628 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.790 INFO:teuthology.orchestra.run.smithi064.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:49.794 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/301 2024-04-01T01:37:49.794 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/301 2024-04-01T01:37:49.794 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:49.796 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:49.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:49.804 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-04-01T01:37:49.982 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:49.986 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:49.986 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:50.133 INFO:teuthology.orchestra.run.smithi064.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:50.137 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/302 2024-04-01T01:37:50.137 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/302 2024-04-01T01:37:50.137 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:50.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:50.147 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-04-01T01:37:50.324 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:50.327 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:50.328 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:50.537 INFO:teuthology.orchestra.run.smithi064.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:37:50.541 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/303 2024-04-01T01:37:50.542 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/303 2024-04-01T01:37:50.542 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:50.543 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:50.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:50.552 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-04-01T01:37:50.998 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:51.002 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:51.002 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:51.173 INFO:teuthology.orchestra.run.smithi064.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:37:51.176 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/304 2024-04-01T01:37:51.176 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/304 2024-04-01T01:37:51.176 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:51.187 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:51.187 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-04-01T01:37:51.609 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:51.613 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/305 2024-04-01T01:37:51.865 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/305 2024-04-01T01:37:51.865 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:51.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:51.875 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:51.875 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-04-01T01:37:52.056 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:52.060 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:52.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:52.340 INFO:teuthology.orchestra.run.smithi064.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:52.343 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/306 2024-04-01T01:37:52.343 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/306 2024-04-01T01:37:52.343 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:52.345 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:52.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:52.364 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-04-01T01:37:52.537 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:52.541 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:52.541 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:52.766 INFO:teuthology.orchestra.run.smithi064.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:37:52.770 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/307 2024-04-01T01:37:52.770 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/307 2024-04-01T01:37:52.770 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:52.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:52.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:52.801 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-04-01T01:37:52.980 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:52.983 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:52.983 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:53.143 INFO:teuthology.orchestra.run.smithi064.stdout:generic/308 1s 2024-04-01T01:37:53.148 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/308 2024-04-01T01:37:53.149 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:53.151 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:53.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:53.159 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-04-01T01:37:53.333 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:53.337 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:53.337 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:54.519 INFO:teuthology.orchestra.run.smithi064.stdout:generic/309 1s 2024-04-01T01:37:54.525 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/309 2024-04-01T01:37:54.525 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:37:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:37:54.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:37:54.536 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-04-01T01:37:54.991 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:37:54.995 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:37:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:38:07.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:38:07.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:38:38.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:38:38.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:39:08.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:39:08.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:39:17.371 INFO:teuthology.orchestra.run.smithi064.stdout:generic/310 82s 2024-04-01T01:39:17.433 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/310 2024-04-01T01:39:17.434 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:17.436 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:17.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:17.451 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-04-01T01:39:17.647 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:17.650 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:17.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:18.363 INFO:teuthology.orchestra.run.smithi064.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:18.370 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/311 2024-04-01T01:39:18.370 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/311 2024-04-01T01:39:18.370 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:18.372 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:18.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:18.385 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-04-01T01:39:18.605 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:18.609 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:18.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:18.824 INFO:teuthology.orchestra.run.smithi064.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:18.827 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/312 2024-04-01T01:39:18.827 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/312 2024-04-01T01:39:18.827 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:18.829 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:18.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:18.838 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-04-01T01:39:19.025 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:19.029 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:23.401 INFO:teuthology.orchestra.run.smithi064.stdout:generic/313 4s 2024-04-01T01:39:23.407 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/313 2024-04-01T01:39:23.407 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:23.410 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:23.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:23.419 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-04-01T01:39:23.608 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:23.611 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:23.611 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:24.083 INFO:teuthology.orchestra.run.smithi064.stdout:generic/314 1s 2024-04-01T01:39:24.088 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/314 2024-04-01T01:39:24.088 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:24.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:24.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:24.099 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-04-01T01:39:24.274 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:24.278 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:24.278 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:25.212 INFO:teuthology.orchestra.run.smithi064.stdout:generic/315 1s 2024-04-01T01:39:25.218 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/315 2024-04-01T01:39:25.219 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:25.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:25.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:25.231 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-04-01T01:39:25.643 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:25.647 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:25.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:26.538 INFO:teuthology.orchestra.run.smithi064.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T01:39:26.541 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/316 2024-04-01T01:39:26.541 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/316 2024-04-01T01:39:26.541 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:26.543 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:26.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:26.553 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-04-01T01:39:26.966 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:26.974 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.103 INFO:teuthology.orchestra.run.smithi064.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:27.106 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/317 2024-04-01T01:39:27.110 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/317 2024-04-01T01:39:27.110 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:27.117 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-04-01T01:39:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:27.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/318 2024-04-01T01:39:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/318 2024-04-01T01:39:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:27.423 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-04-01T01:39:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:27.600 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.762 INFO:teuthology.orchestra.run.smithi064.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T01:39:27.766 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/319 2024-04-01T01:39:27.766 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/319 2024-04-01T01:39:27.766 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:27.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:27.776 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-04-01T01:39:27.954 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:27.958 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.087 INFO:teuthology.orchestra.run.smithi064.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:28.090 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/320 2024-04-01T01:39:28.091 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/320 2024-04-01T01:39:28.091 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:28.101 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-04-01T01:39:28.275 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:28.279 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.404 INFO:teuthology.orchestra.run.smithi064.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:28.407 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/321 2024-04-01T01:39:28.407 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/321 2024-04-01T01:39:28.407 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:28.418 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-04-01T01:39:28.588 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:28.592 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.717 INFO:teuthology.orchestra.run.smithi064.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:39:28.721 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/322 2024-04-01T01:39:28.721 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/322 2024-04-01T01:39:28.721 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:39:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:28.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:39:28.731 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-04-01T01:39:28.907 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:39:28.911 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:39:28.911 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:39:38.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:39:38.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:40:08.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:40:08.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:40:08.617 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T01:40:08.617+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47265) UID: 0 2024-04-01T01:40:08.617 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T01:40:08.617+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47265) UID: 0 2024-04-01T01:40:08.618 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T01:40:08.617+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47265) UID: 0 2024-04-01T01:40:08.618 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T01:40:08.617+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47265) UID: 0 2024-04-01T01:41:29.610 INFO:teuthology.orchestra.run.smithi064.stdout:generic/323 121s 2024-04-01T01:41:29.615 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/323 2024-04-01T01:41:29.615 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:29.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:29.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:29.632 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-04-01T01:41:30.034 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:30.038 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:30.038 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.168 INFO:teuthology.orchestra.run.smithi064.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:30.172 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/324 2024-04-01T01:41:30.172 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/324 2024-04-01T01:41:30.172 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:30.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.182 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:30.182 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-04-01T01:41:30.351 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:30.355 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:30.355 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.496 INFO:teuthology.orchestra.run.smithi064.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:30.499 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/325 2024-04-01T01:41:30.499 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/325 2024-04-01T01:41:30.499 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:30.501 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:30.510 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-04-01T01:41:30.685 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:30.689 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:30.689 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.823 INFO:teuthology.orchestra.run.smithi064.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:30.826 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/326 2024-04-01T01:41:30.826 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/326 2024-04-01T01:41:30.826 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:30.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:30.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:30.837 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-04-01T01:41:31.011 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:31.014 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.126 INFO:teuthology.orchestra.run.smithi064.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:31.129 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/327 2024-04-01T01:41:31.129 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/327 2024-04-01T01:41:31.129 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:31.131 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:31.140 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-04-01T01:41:31.310 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:31.314 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:31.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.433 INFO:teuthology.orchestra.run.smithi064.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:31.436 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/328 2024-04-01T01:41:31.436 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/328 2024-04-01T01:41:31.437 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:31.439 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:31.447 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-04-01T01:41:31.617 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:31.621 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:31.621 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.750 INFO:teuthology.orchestra.run.smithi064.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:31.754 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/329 2024-04-01T01:41:31.754 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/329 2024-04-01T01:41:31.754 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:31.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:31.765 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-04-01T01:41:31.938 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:31.942 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:31.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.058 INFO:teuthology.orchestra.run.smithi064.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:32.061 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/330 2024-04-01T01:41:32.061 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/330 2024-04-01T01:41:32.061 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:32.063 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:32.072 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-04-01T01:41:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.382 INFO:teuthology.orchestra.run.smithi064.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:32.386 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/331 2024-04-01T01:41:32.386 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/331 2024-04-01T01:41:32.386 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.396 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:32.396 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-04-01T01:41:32.571 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.691 INFO:teuthology.orchestra.run.smithi064.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:32.694 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/332 2024-04-01T01:41:32.694 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/332 2024-04-01T01:41:32.694 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:32.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:32.706 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-04-01T01:41:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:32.885 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.021 INFO:teuthology.orchestra.run.smithi064.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:33.024 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/333 2024-04-01T01:41:33.024 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/333 2024-04-01T01:41:33.024 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:33.035 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-04-01T01:41:33.211 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:33.215 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.332 INFO:teuthology.orchestra.run.smithi064.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:33.336 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/334 2024-04-01T01:41:33.336 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/334 2024-04-01T01:41:33.336 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:33.346 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-04-01T01:41:33.518 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:33.521 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.645 INFO:teuthology.orchestra.run.smithi064.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:33.649 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/335 2024-04-01T01:41:33.649 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/335 2024-04-01T01:41:33.649 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:33.660 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-04-01T01:41:33.832 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:33.835 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.960 INFO:teuthology.orchestra.run.smithi064.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:33.964 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/336 2024-04-01T01:41:33.964 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/336 2024-04-01T01:41:33.964 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:33.966 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:33.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:33.974 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-04-01T01:41:34.146 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:34.149 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:34.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.268 INFO:teuthology.orchestra.run.smithi064.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:34.271 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/337 2024-04-01T01:41:34.271 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/337 2024-04-01T01:41:34.271 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:34.274 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:34.282 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-04-01T01:41:34.455 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:34.458 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:34.458 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.584 INFO:teuthology.orchestra.run.smithi064.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:34.587 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/338 2024-04-01T01:41:34.587 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/338 2024-04-01T01:41:34.587 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:34.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:34.598 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-04-01T01:41:34.776 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:34.780 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.899 INFO:teuthology.orchestra.run.smithi064.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:34.902 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/339 2024-04-01T01:41:34.903 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/339 2024-04-01T01:41:34.903 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:34.905 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:34.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:34.913 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-04-01T01:41:35.093 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:35.097 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:35.097 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.236 INFO:teuthology.orchestra.run.smithi064.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:35.239 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/340 2024-04-01T01:41:35.239 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/340 2024-04-01T01:41:35.239 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:35.241 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:35.250 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-04-01T01:41:35.425 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:35.429 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:35.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:35.558 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/341 2024-04-01T01:41:35.558 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/341 2024-04-01T01:41:35.558 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:35.560 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:35.569 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-04-01T01:41:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:35.744 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.878 INFO:teuthology.orchestra.run.smithi064.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:35.881 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/342 2024-04-01T01:41:35.881 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/342 2024-04-01T01:41:35.881 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:35.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:35.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:35.892 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-04-01T01:41:36.065 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:36.069 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.197 INFO:teuthology.orchestra.run.smithi064.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:36.200 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/343 2024-04-01T01:41:36.200 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/343 2024-04-01T01:41:36.200 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:36.211 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-04-01T01:41:36.388 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:36.392 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:36.392 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.504 INFO:teuthology.orchestra.run.smithi064.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:36.508 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/344 2024-04-01T01:41:36.508 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/344 2024-04-01T01:41:36.508 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:36.536 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-04-01T01:41:36.708 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:36.712 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:36.712 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.825 INFO:teuthology.orchestra.run.smithi064.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:36.829 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/345 2024-04-01T01:41:36.829 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/345 2024-04-01T01:41:36.829 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:36.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:36.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:36.839 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-04-01T01:41:37.008 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:37.012 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.128 INFO:teuthology.orchestra.run.smithi064.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:37.132 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/346 2024-04-01T01:41:37.132 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/346 2024-04-01T01:41:37.132 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:37.143 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-04-01T01:41:37.324 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:37.328 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:37.328 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.484 INFO:teuthology.orchestra.run.smithi064.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:37.488 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/347 2024-04-01T01:41:37.488 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/347 2024-04-01T01:41:37.488 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:37.490 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:37.506 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-04-01T01:41:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:37.687 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:37.687 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.812 INFO:teuthology.orchestra.run.smithi064.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:37.815 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/348 2024-04-01T01:41:37.816 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/348 2024-04-01T01:41:37.816 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:37.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:37.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:37.827 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-04-01T01:41:38.010 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:38.014 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:38.014 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:38.885 INFO:teuthology.orchestra.run.smithi064.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-01T01:41:38.889 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/349 2024-04-01T01:41:38.889 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/349 2024-04-01T01:41:38.889 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:38.891 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:38.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:38.908 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-04-01T01:41:39.078 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:39.082 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:39.082 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:40.386 INFO:teuthology.orchestra.run.smithi064.stdout:generic/350 1s 2024-04-01T01:41:40.391 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/350 2024-04-01T01:41:40.391 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:40.393 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:40.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:40.402 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-04-01T01:41:40.582 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:40.586 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:40.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:41.474 INFO:teuthology.orchestra.run.smithi064.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-01T01:41:41.477 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/351 2024-04-01T01:41:41.477 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/351 2024-04-01T01:41:41.477 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:41.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:41.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:41.533 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-04-01T01:41:41.887 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:41.891 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.021 INFO:teuthology.orchestra.run.smithi064.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:42.024 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/352 2024-04-01T01:41:42.024 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/352 2024-04-01T01:41:42.025 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:42.036 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-04-01T01:41:42.209 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:42.213 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.335 INFO:teuthology.orchestra.run.smithi064.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:42.338 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/353 2024-04-01T01:41:42.339 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/353 2024-04-01T01:41:42.339 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:42.349 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-04-01T01:41:42.522 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:42.526 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.652 INFO:teuthology.orchestra.run.smithi064.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:42.656 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/354 2024-04-01T01:41:42.656 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/354 2024-04-01T01:41:42.656 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:42.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:42.675 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-04-01T01:41:42.847 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:42.851 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:43.435 INFO:teuthology.orchestra.run.smithi064.stdout:generic/355 1s 2024-04-01T01:41:43.440 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/355 2024-04-01T01:41:43.440 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:43.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:43.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:43.451 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-04-01T01:41:43.848 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:43.852 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:43.977 INFO:teuthology.orchestra.run.smithi064.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:43.980 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/356 2024-04-01T01:41:43.981 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/356 2024-04-01T01:41:43.981 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:43.983 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:43.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:43.992 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-04-01T01:41:44.163 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:44.167 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:44.167 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:44.283 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/357 2024-04-01T01:41:44.284 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/357 2024-04-01T01:41:44.284 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:44.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:44.295 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-04-01T01:41:44.475 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:44.478 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:44.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.598 INFO:teuthology.orchestra.run.smithi064.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:44.602 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/358 2024-04-01T01:41:44.602 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/358 2024-04-01T01:41:44.602 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:44.604 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:44.613 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-04-01T01:41:44.788 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:44.792 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:44.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.907 INFO:teuthology.orchestra.run.smithi064.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:44.910 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/359 2024-04-01T01:41:44.910 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/359 2024-04-01T01:41:44.910 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:44.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:44.922 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-04-01T01:41:45.096 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:45.100 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:45.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.285 INFO:teuthology.orchestra.run.smithi064.stdout:generic/360 0s 2024-04-01T01:41:45.289 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/360 2024-04-01T01:41:45.289 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:45.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:45.301 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-04-01T01:41:45.481 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:45.485 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:45.485 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.630 INFO:teuthology.orchestra.run.smithi064.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:45.634 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/361 2024-04-01T01:41:45.634 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/361 2024-04-01T01:41:45.634 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:45.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.644 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:45.645 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-04-01T01:41:45.819 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:45.823 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:45.823 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.942 INFO:teuthology.orchestra.run.smithi064.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:45.945 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/362 2024-04-01T01:41:45.946 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/362 2024-04-01T01:41:45.946 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:45.948 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:45.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:45.956 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-04-01T01:41:46.129 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:46.133 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:46.133 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.252 INFO:teuthology.orchestra.run.smithi064.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:46.255 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/363 2024-04-01T01:41:46.255 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/363 2024-04-01T01:41:46.255 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:46.257 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:46.265 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-04-01T01:41:46.449 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:46.454 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.575 INFO:teuthology.orchestra.run.smithi064.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:46.578 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/364 2024-04-01T01:41:46.578 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/364 2024-04-01T01:41:46.578 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:46.580 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:46.589 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-04-01T01:41:46.761 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:46.765 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.892 INFO:teuthology.orchestra.run.smithi064.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:46.895 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/365 2024-04-01T01:41:46.896 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/365 2024-04-01T01:41:46.896 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:46.898 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:46.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:46.906 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-04-01T01:41:47.079 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:47.083 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:47.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:47.195 INFO:teuthology.orchestra.run.smithi064.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:47.198 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/366 2024-04-01T01:41:47.198 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/366 2024-04-01T01:41:47.198 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:47.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:47.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:47.225 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-04-01T01:41:47.395 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:47.398 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:47.399 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/367 2024-04-01T01:41:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/367 2024-04-01T01:41:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:47.595 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:47.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:47.603 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-04-01T01:41:47.793 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:47.797 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:47.797 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.001 INFO:teuthology.orchestra.run.smithi064.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:48.005 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/368 2024-04-01T01:41:48.005 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/368 2024-04-01T01:41:48.005 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:48.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:48.016 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-04-01T01:41:48.189 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:48.193 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.384 INFO:teuthology.orchestra.run.smithi064.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:48.388 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/369 2024-04-01T01:41:48.388 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/369 2024-04-01T01:41:48.388 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:48.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:48.398 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-04-01T01:41:48.569 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:48.573 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:48.573 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.776 INFO:teuthology.orchestra.run.smithi064.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:48.779 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/370 2024-04-01T01:41:48.780 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/370 2024-04-01T01:41:48.780 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:48.782 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:48.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:48.790 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-04-01T01:41:48.965 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:48.969 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:48.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.132 INFO:teuthology.orchestra.run.smithi064.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:49.135 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/371 2024-04-01T01:41:49.135 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/371 2024-04-01T01:41:49.135 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:49.146 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-04-01T01:41:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:49.318 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.436 INFO:teuthology.orchestra.run.smithi064.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:49.439 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/372 2024-04-01T01:41:49.439 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/372 2024-04-01T01:41:49.439 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:49.441 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:49.450 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-04-01T01:41:49.629 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:49.633 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.752 INFO:teuthology.orchestra.run.smithi064.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:49.756 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/373 2024-04-01T01:41:49.756 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/373 2024-04-01T01:41:49.756 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:49.758 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:49.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:49.783 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-04-01T01:41:49.971 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:49.974 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:49.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.097 INFO:teuthology.orchestra.run.smithi064.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:50.101 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/374 2024-04-01T01:41:50.101 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/374 2024-04-01T01:41:50.101 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:50.111 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-04-01T01:41:50.287 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:50.291 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:50.291 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.455 INFO:teuthology.orchestra.run.smithi064.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T01:41:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/375 2024-04-01T01:41:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/375 2024-04-01T01:41:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:50.460 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:50.469 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-04-01T01:41:50.648 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:50.652 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.780 INFO:teuthology.orchestra.run.smithi064.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:50.784 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/376 2024-04-01T01:41:50.784 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/376 2024-04-01T01:41:50.784 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:50.786 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:50.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:50.805 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-04-01T01:41:50.979 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:50.983 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:50.983 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:51.109 INFO:teuthology.orchestra.run.smithi064.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:51.112 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/377 2024-04-01T01:41:51.112 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/377 2024-04-01T01:41:51.112 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:51.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:51.122 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-04-01T01:41:51.302 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:51.307 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:51.310 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:generic/378 0s 2024-04-01T01:41:51.550 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/378 2024-04-01T01:41:51.550 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:51.553 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:51.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:51.561 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-04-01T01:41:51.977 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:51.980 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:51.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.109 INFO:teuthology.orchestra.run.smithi064.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:52.112 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/379 2024-04-01T01:41:52.113 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/379 2024-04-01T01:41:52.113 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:52.114 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:52.123 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-04-01T01:41:52.308 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:52.312 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:52.312 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.438 INFO:teuthology.orchestra.run.smithi064.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:52.442 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/380 2024-04-01T01:41:52.442 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/380 2024-04-01T01:41:52.442 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:52.444 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:52.464 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-04-01T01:41:52.638 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:52.642 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:52.642 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.756 INFO:teuthology.orchestra.run.smithi064.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:52.760 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/381 2024-04-01T01:41:52.760 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/381 2024-04-01T01:41:52.760 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:52.762 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:52.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:52.771 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-04-01T01:41:52.951 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:52.955 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:52.955 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.075 INFO:teuthology.orchestra.run.smithi064.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:53.078 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/382 2024-04-01T01:41:53.078 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/382 2024-04-01T01:41:53.078 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:53.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:53.088 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-04-01T01:41:53.259 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:53.263 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:53.263 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.396 INFO:teuthology.orchestra.run.smithi064.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:53.399 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/383 2024-04-01T01:41:53.399 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/383 2024-04-01T01:41:53.399 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:53.402 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:53.410 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-04-01T01:41:53.582 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:53.586 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:53.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.710 INFO:teuthology.orchestra.run.smithi064.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T01:41:53.713 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/384 2024-04-01T01:41:53.713 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/384 2024-04-01T01:41:53.713 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:53.715 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:53.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:53.724 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-04-01T01:41:53.901 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:53.904 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.029 INFO:teuthology.orchestra.run.smithi064.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:54.033 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/385 2024-04-01T01:41:54.033 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/385 2024-04-01T01:41:54.033 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:54.035 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:54.043 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-04-01T01:41:54.217 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:54.221 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.338 INFO:teuthology.orchestra.run.smithi064.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T01:41:54.342 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/386 2024-04-01T01:41:54.342 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/386 2024-04-01T01:41:54.342 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:54.352 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-04-01T01:41:54.526 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:54.530 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.644 INFO:teuthology.orchestra.run.smithi064.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:54.647 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/387 2024-04-01T01:41:54.648 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/387 2024-04-01T01:41:54.648 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:54.658 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-04-01T01:41:54.834 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:54.838 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.956 INFO:teuthology.orchestra.run.smithi064.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/388 2024-04-01T01:41:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/388 2024-04-01T01:41:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:54.982 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:54.982 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-04-01T01:41:55.159 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:55.163 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:55.163 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:55.292 INFO:teuthology.orchestra.run.smithi064.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-04-01T01:41:55.296 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/389 2024-04-01T01:41:55.296 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/389 2024-04-01T01:41:55.296 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:55.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:55.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:55.306 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-04-01T01:41:55.484 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:55.488 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:55.488 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:55.624 INFO:teuthology.orchestra.run.smithi064.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:41:55.627 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/390 2024-04-01T01:41:55.627 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/390 2024-04-01T01:41:55.627 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:41:55.629 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:41:55.638 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:41:55.638 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-04-01T01:41:55.819 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:41:55.823 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:41:55.823 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:15.131 INFO:teuthology.orchestra.run.smithi064.stdout:generic/391 20s 2024-04-01T01:42:15.190 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/391 2024-04-01T01:42:15.190 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:15.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:15.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:15.399 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-04-01T01:42:16.219 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:16.224 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:16.224 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:16.413 INFO:teuthology.orchestra.run.smithi064.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:16.416 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/392 2024-04-01T01:42:16.416 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/392 2024-04-01T01:42:16.416 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:16.419 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:16.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:16.430 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-04-01T01:42:16.669 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:16.675 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:16.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:16.812 INFO:teuthology.orchestra.run.smithi064.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:16.816 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/393 2024-04-01T01:42:16.817 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/393 2024-04-01T01:42:16.817 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:16.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:16.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:16.830 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-04-01T01:42:17.096 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:17.101 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:17.353 INFO:teuthology.orchestra.run.smithi064.stdout:generic/394 0s 2024-04-01T01:42:17.359 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/394 2024-04-01T01:42:17.359 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:17.362 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:17.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:17.370 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-04-01T01:42:17.882 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:17.886 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:17.886 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.070 INFO:teuthology.orchestra.run.smithi064.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:18.075 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/395 2024-04-01T01:42:18.075 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/395 2024-04-01T01:42:18.075 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:18.089 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-04-01T01:42:18.322 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:18.328 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:18.329 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.509 INFO:teuthology.orchestra.run.smithi064.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:18.513 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/396 2024-04-01T01:42:18.513 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/396 2024-04-01T01:42:18.513 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:18.526 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-04-01T01:42:18.756 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:18.761 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:18.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.961 INFO:teuthology.orchestra.run.smithi064.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:18.964 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/397 2024-04-01T01:42:18.965 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/397 2024-04-01T01:42:18.965 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:18.967 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:18.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:18.976 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-04-01T01:42:19.189 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:19.193 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:19.193 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:19.373 INFO:teuthology.orchestra.run.smithi064.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:19.377 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/398 2024-04-01T01:42:19.377 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/398 2024-04-01T01:42:19.377 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:19.379 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:19.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:19.388 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-04-01T01:42:19.631 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:19.636 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:19.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:19.818 INFO:teuthology.orchestra.run.smithi064.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:19.822 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/399 2024-04-01T01:42:19.822 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/399 2024-04-01T01:42:19.822 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:19.824 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:19.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:19.836 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-04-01T01:42:20.057 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:20.061 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:20.061 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:20.204 INFO:teuthology.orchestra.run.smithi064.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T01:42:20.209 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/400 2024-04-01T01:42:20.209 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/400 2024-04-01T01:42:20.209 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:20.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:20.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:20.227 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-04-01T01:42:20.476 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:20.484 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:20.484 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:20.637 INFO:teuthology.orchestra.run.smithi064.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:20.642 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/401 2024-04-01T01:42:20.642 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/401 2024-04-01T01:42:20.642 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:20.645 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:20.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:20.655 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-04-01T01:42:20.890 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:20.894 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:20.894 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:21.040 INFO:teuthology.orchestra.run.smithi064.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:21.044 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/402 2024-04-01T01:42:21.044 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/402 2024-04-01T01:42:21.044 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:21.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:21.054 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-04-01T01:42:21.225 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:21.229 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:21.229 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:21.340 INFO:teuthology.orchestra.run.smithi064.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:21.344 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/403 2024-04-01T01:42:21.344 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/403 2024-04-01T01:42:21.344 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:21.346 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:21.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:21.355 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-04-01T01:42:21.526 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:21.530 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-01T01:42:22.404 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/404 2024-04-01T01:42:22.404 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/404 2024-04-01T01:42:22.404 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:22.406 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:22.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:22.449 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-04-01T01:42:22.837 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:22.842 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:22.842 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.036 INFO:teuthology.orchestra.run.smithi064.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:23.040 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/405 2024-04-01T01:42:23.040 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/405 2024-04-01T01:42:23.040 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:23.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:23.051 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-04-01T01:42:23.220 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:23.223 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:23.224 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.359 INFO:teuthology.orchestra.run.smithi064.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:23.362 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/406 2024-04-01T01:42:23.362 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/406 2024-04-01T01:42:23.362 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:23.364 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:23.373 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-04-01T01:42:23.545 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:23.548 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:23.549 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:42:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/407 2024-04-01T01:42:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/407 2024-04-01T01:42:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:23.734 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:23.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:23.742 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-04-01T01:42:24.193 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:24.197 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:24.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:24.375 INFO:teuthology.orchestra.run.smithi064.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:42:24.378 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/408 2024-04-01T01:42:24.378 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/408 2024-04-01T01:42:24.379 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:24.381 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:24.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:24.389 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-04-01T01:42:24.804 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:24.809 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:24.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/409 2024-04-01T01:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/409 2024-04-01T01:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:24.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:24.947 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-04-01T01:42:25.126 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:25.129 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:25.130 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.254 INFO:teuthology.orchestra.run.smithi064.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:25.257 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/410 2024-04-01T01:42:25.257 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/410 2024-04-01T01:42:25.257 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:25.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:25.267 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-04-01T01:42:25.449 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:25.453 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:25.453 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.588 INFO:teuthology.orchestra.run.smithi064.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:25.591 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/411 2024-04-01T01:42:25.592 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/411 2024-04-01T01:42:25.592 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:25.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:25.602 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-04-01T01:42:25.785 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:25.790 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:25.790 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.917 INFO:teuthology.orchestra.run.smithi064.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:25.921 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/412 2024-04-01T01:42:25.921 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/412 2024-04-01T01:42:25.921 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:25.923 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:25.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:25.932 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-04-01T01:42:26.102 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:26.106 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:26.106 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.226 INFO:teuthology.orchestra.run.smithi064.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:26.230 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/413 2024-04-01T01:42:26.230 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/413 2024-04-01T01:42:26.230 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:26.232 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:26.241 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-04-01T01:42:26.420 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:26.424 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:26.424 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.536 INFO:teuthology.orchestra.run.smithi064.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:26.539 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/414 2024-04-01T01:42:26.539 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/414 2024-04-01T01:42:26.539 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:26.541 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:26.550 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-04-01T01:42:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:26.734 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.856 INFO:teuthology.orchestra.run.smithi064.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:26.859 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/415 2024-04-01T01:42:26.860 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/415 2024-04-01T01:42:26.860 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:26.862 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:26.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:26.870 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-04-01T01:42:27.040 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:27.044 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:27.159 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/416 2024-04-01T01:42:27.159 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/416 2024-04-01T01:42:27.159 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:27.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:27.170 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-04-01T01:42:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/417 2024-04-01T01:42:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/417 2024-04-01T01:42:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:27.477 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-04-01T01:42:27.649 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:27.653 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.806 INFO:teuthology.orchestra.run.smithi064.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2024-04-01T01:42:27.810 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/418 2024-04-01T01:42:27.810 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/418 2024-04-01T01:42:27.810 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:27.820 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:27.820 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-04-01T01:42:27.991 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:27.994 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:28.134 INFO:teuthology.orchestra.run.smithi064.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:28.137 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/419 2024-04-01T01:42:28.137 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/419 2024-04-01T01:42:28.137 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:28.139 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:28.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:28.148 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-04-01T01:42:28.323 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:28.327 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:29.201 INFO:teuthology.orchestra.run.smithi064.stdout:generic/420 1s 2024-04-01T01:42:29.206 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/420 2024-04-01T01:42:29.206 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:29.208 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:29.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:29.228 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-04-01T01:42:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:29.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:29.791 INFO:teuthology.orchestra.run.smithi064.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:29.794 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/421 2024-04-01T01:42:29.794 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/421 2024-04-01T01:42:29.794 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:29.796 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:29.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:29.820 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-04-01T01:42:30.003 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:30.007 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:30.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/422 2024-04-01T01:42:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/422 2024-04-01T01:42:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:30.129 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:30.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:30.138 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-04-01T01:42:30.314 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:30.318 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:30.318 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:30.555 INFO:teuthology.orchestra.run.smithi064.stdout:generic/423 0s 2024-04-01T01:42:30.560 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/423 2024-04-01T01:42:30.560 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:30.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:30.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:30.571 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-04-01T01:42:31.036 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:31.040 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:31.198 INFO:teuthology.orchestra.run.smithi064.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-04-01T01:42:31.201 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/424 2024-04-01T01:42:31.201 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/424 2024-04-01T01:42:31.201 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:31.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:31.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:31.212 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-04-01T01:42:31.395 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:31.399 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:31.399 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:31.514 INFO:teuthology.orchestra.run.smithi064.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:42:31.518 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/425 2024-04-01T01:42:31.518 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/425 2024-04-01T01:42:31.518 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:42:31.519 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:42:31.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:42:31.527 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-04-01T01:42:31.697 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:42:31.701 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:42:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:03.112 INFO:teuthology.orchestra.run.smithi064.stdout:generic/426 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/426.out.bad) 2024-04-01T01:43:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/426.out 2024-04-01 00:14:57.021907963 +0000 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/426.out.bad 2024-04-01 01:43:03.087796485 +0000 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,5 +1,947 @@ 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 426 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/426-dir -d 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000082) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000083) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000084) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000085) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000086) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000087) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000088) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000089) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000090) opened an unlinked file! 2024-04-01T01:43:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000091) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000092) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000093) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000094) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000095) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000096) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000097) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000098) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000099) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000100) opened an unlinked file! 2024-04-01T01:43:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000101) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000102) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000103) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000104) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000105) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000106) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000107) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000108) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000109) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000110) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000111) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000112) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000113) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000114) opened an unlinked file! 2024-04-01T01:43:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000115) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000116) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000117) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000118) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000119) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000120) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000121) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000122) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000123) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000124) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000125) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000126) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000127) opened an unlinked file! 2024-04-01T01:43:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000128) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000129) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000130) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000131) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000132) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000133) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000134) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000135) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000136) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000137) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000138) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000139) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000140) opened an unlinked file! 2024-04-01T01:43:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000141) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000142) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000143) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000144) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000145) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000146) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000147) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000148) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000149) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000150) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000151) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000152) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000153) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000154) opened an unlinked file! 2024-04-01T01:43:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000155) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000156) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000157) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000158) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000159) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000160) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000161) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000162) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000163) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000164) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000165) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000166) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000167) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000168) opened an unlinked file! 2024-04-01T01:43:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000169) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000170) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000171) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000172) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000173) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000174) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000175) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000176) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000177) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000178) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000179) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000180) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000181) opened an unlinked file! 2024-04-01T01:43:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000182) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000183) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000184) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000185) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000186) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000187) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000188) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000189) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000190) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000191) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000192) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000193) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000194) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000195) opened an unlinked file! 2024-04-01T01:43:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000196) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000197) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000198) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000199) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000200) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000201) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000202) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000203) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000204) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000205) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000206) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000207) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000208) opened an unlinked file! 2024-04-01T01:43:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000209) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000210) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000211) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000212) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000213) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000214) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000215) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000216) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000217) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000218) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000219) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000220) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000221) opened an unlinked file! 2024-04-01T01:43:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000222) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000223) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000224) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000225) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000226) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000227) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000228) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000229) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000230) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000231) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000232) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000233) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000234) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000235) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000236) opened an unlinked file! 2024-04-01T01:43:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000237) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000238) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000239) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000240) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000241) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000242) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000243) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000244) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000245) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000246) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000247) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000248) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000249) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000250) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000251) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000252) opened an unlinked file! 2024-04-01T01:43:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000253) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000254) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000255) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000256) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000257) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000258) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000259) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000260) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000261) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000262) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000263) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000264) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000265) opened an unlinked file! 2024-04-01T01:43:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000266) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000267) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000268) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000269) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000270) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000271) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000272) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000273) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000274) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000275) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000276) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000277) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000278) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000279) opened an unlinked file! 2024-04-01T01:43:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000280) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000281) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000282) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000283) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000284) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000285) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000286) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000287) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000288) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000289) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000290) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000291) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000292) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000293) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000294) opened an unlinked file! 2024-04-01T01:43:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000295) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000296) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000297) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000298) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000299) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000300) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000301) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000302) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000303) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000304) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000305) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000306) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000307) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000308) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000309) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000310) opened an unlinked file! 2024-04-01T01:43:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000311) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000312) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000313) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000314) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000315) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000316) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000317) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000318) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000319) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000320) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000321) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000322) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000323) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000324) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000325) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000326) opened an unlinked file! 2024-04-01T01:43:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000327) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000328) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000329) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000330) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000331) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000332) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000333) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000334) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000335) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000336) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000337) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000338) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000339) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000340) opened an unlinked file! 2024-04-01T01:43:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000341) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000342) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000343) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000344) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000345) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000346) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000347) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000348) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000349) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000350) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000351) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000352) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000353) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000354) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000355) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000356) opened an unlinked file! 2024-04-01T01:43:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000357) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000358) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000359) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000360) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000361) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000362) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000363) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000364) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000365) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000366) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000367) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000368) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000369) opened an unlinked file! 2024-04-01T01:43:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000370) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000371) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000372) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000373) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000374) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000375) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000376) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000377) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000378) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000379) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000380) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000381) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000382) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000383) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000384) opened an unlinked file! 2024-04-01T01:43:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000385) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000386) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000387) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000388) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000389) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000390) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000391) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000392) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000393) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000394) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000395) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000396) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000397) opened an unlinked file! 2024-04-01T01:43:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000398) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000399) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000400) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000401) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000402) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000403) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000404) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000405) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000406) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000407) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000408) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000409) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000410) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000411) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000412) opened an unlinked file! 2024-04-01T01:43:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000413) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000414) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000415) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000416) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000417) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000418) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000419) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000420) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000421) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000422) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000423) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000424) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000425) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000426) opened an unlinked file! 2024-04-01T01:43:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000427) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000428) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000429) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000430) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000431) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000432) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000433) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000434) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000435) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000436) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000437) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000438) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000439) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000440) opened an unlinked file! 2024-04-01T01:43:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000441) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000442) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000443) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000444) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000445) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000446) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000447) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000448) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000449) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000450) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000451) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000452) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000453) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000454) opened an unlinked file! 2024-04-01T01:43:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000455) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000456) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000457) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000458) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000459) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000460) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000461) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000462) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000463) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000464) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000465) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000466) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000467) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000468) opened an unlinked file! 2024-04-01T01:43:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000469) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000470) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000471) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000472) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000473) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000474) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000475) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000476) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000477) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000478) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000479) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000480) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000481) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000482) opened an unlinked file! 2024-04-01T01:43:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000483) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000484) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000485) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000486) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000487) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000488) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000489) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000490) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000491) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000492) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000493) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000494) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000495) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000496) opened an unlinked file! 2024-04-01T01:43:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000497) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000498) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000499) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000500) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000501) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000502) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000503) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000504) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000505) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000506) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000507) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000508) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000509) opened an unlinked file! 2024-04-01T01:43:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000510) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000511) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000512) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000513) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000514) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000515) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000516) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000517) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000518) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000519) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000520) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000521) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000522) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000523) opened an unlinked file! 2024-04-01T01:43:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000524) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000525) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000526) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000527) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000528) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000529) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000530) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000531) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000532) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000533) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000534) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000535) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000536) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000537) opened an unlinked file! 2024-04-01T01:43:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000538) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000539) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000540) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000541) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000542) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000543) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000544) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000545) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000546) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000547) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000548) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000549) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000550) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000551) opened an unlinked file! 2024-04-01T01:43:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000552) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000553) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000554) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000555) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000556) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000557) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000558) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000559) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000560) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000561) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000562) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000563) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000564) opened an unlinked file! 2024-04-01T01:43:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000565) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000566) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000567) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000568) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000569) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000570) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000571) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000572) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000573) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000574) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000575) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000576) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000577) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000578) opened an unlinked file! 2024-04-01T01:43:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000579) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000580) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000581) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000582) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000583) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000584) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000585) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000586) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000587) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000588) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000589) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000590) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000591) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000592) opened an unlinked file! 2024-04-01T01:43:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000593) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000594) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000595) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000596) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000597) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000598) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000599) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000600) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000601) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000602) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000603) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000604) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000605) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000606) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000607) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000608) opened an unlinked file! 2024-04-01T01:43:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000609) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000610) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000611) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000612) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000613) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000614) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000615) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000616) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000617) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000618) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000619) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000620) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000621) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000622) opened an unlinked file! 2024-04-01T01:43:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000623) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000624) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000625) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000626) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000627) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000628) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000629) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000630) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000631) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000632) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000633) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000634) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000635) opened an unlinked file! 2024-04-01T01:43:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000636) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000637) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000638) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000639) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000640) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000641) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000642) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000643) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000644) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000645) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000646) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000647) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000648) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000649) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000650) opened an unlinked file! 2024-04-01T01:43:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000651) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000652) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000653) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000654) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000655) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000656) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000657) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000658) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000659) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000660) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000661) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000662) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000663) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000664) opened an unlinked file! 2024-04-01T01:43:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000665) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000666) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000667) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000668) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000669) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000670) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000671) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000672) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000673) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000674) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000675) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000676) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000677) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000678) opened an unlinked file! 2024-04-01T01:43:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000679) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000680) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000681) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000682) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000683) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000684) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000685) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000686) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000687) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000688) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000689) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000690) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000691) opened an unlinked file! 2024-04-01T01:43:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000692) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000693) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000694) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000695) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000696) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000697) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000698) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000699) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000700) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000701) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000702) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000703) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000704) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000705) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000706) opened an unlinked file! 2024-04-01T01:43:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000707) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000708) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000709) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000710) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000711) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000712) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000713) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000714) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000715) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000716) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000717) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000718) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000719) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000720) opened an unlinked file! 2024-04-01T01:43:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000721) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000722) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000723) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000724) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000725) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000726) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000727) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000728) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000729) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000730) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000731) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000732) opened an unlinked file! 2024-04-01T01:43:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000733) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000734) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000735) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000736) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000737) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000738) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000739) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000740) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000741) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000742) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000743) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000744) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000745) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000746) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000747) opened an unlinked file! 2024-04-01T01:43:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000748) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000749) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000750) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000751) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000752) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000753) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000754) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000755) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000756) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000757) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000758) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000759) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000760) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000761) opened an unlinked file! 2024-04-01T01:43:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000762) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000763) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000764) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000765) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000766) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000767) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000768) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000769) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000770) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000771) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000772) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000773) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000774) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000775) opened an unlinked file! 2024-04-01T01:43:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000776) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000777) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000778) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000779) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000780) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000781) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000782) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000783) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000784) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000785) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000786) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000787) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000788) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000789) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000790) opened an unlinked file! 2024-04-01T01:43:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000791) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000792) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000793) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000794) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000795) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000796) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000797) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000798) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000799) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000800) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000801) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000802) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000803) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000804) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000805) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000806) opened an unlinked file! 2024-04-01T01:43:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000807) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000808) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000809) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000810) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000811) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000812) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000813) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000814) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000815) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000816) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000817) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000818) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000819) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000820) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000821) opened an unlinked file! 2024-04-01T01:43:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000822) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000823) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000824) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000825) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000826) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000827) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000828) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000829) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000830) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000831) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000832) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000833) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000834) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000835) opened an unlinked file! 2024-04-01T01:43:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000836) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000837) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000838) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000839) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000840) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000841) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000842) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000843) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000844) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000845) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000846) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000847) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000848) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000849) opened an unlinked file! 2024-04-01T01:43:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000850) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000851) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000852) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000853) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000854) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000855) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000856) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000857) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000858) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000859) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000860) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000861) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000862) opened an unlinked file! 2024-04-01T01:43:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000863) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000864) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000865) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000866) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000867) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000868) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000869) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000870) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000871) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000872) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000873) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000874) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000875) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000876) opened an unlinked file! 2024-04-01T01:43:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000877) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000878) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000879) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000880) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000881) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000882) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000883) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000884) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000885) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000886) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000887) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000888) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000889) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000890) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000891) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000892) opened an unlinked file! 2024-04-01T01:43:03.185 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000893) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000894) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000895) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000896) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000897) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000898) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000899) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000900) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000901) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000902) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000903) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000904) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000905) opened an unlinked file! 2024-04-01T01:43:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000906) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000907) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000908) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000909) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000910) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000911) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000912) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000913) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000914) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000915) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000916) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000917) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000918) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000919) opened an unlinked file! 2024-04-01T01:43:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000920) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000921) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000922) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000923) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000924) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000925) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000926) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000927) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000928) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000929) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000930) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000931) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000932) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000933) opened an unlinked file! 2024-04-01T01:43:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000934) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000935) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000936) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000937) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000938) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000939) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000940) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000941) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000942) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000943) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000944) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000945) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000946) opened an unlinked file! 2024-04-01T01:43:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000947) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000948) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000949) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000950) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000951) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000952) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000953) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000954) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000955) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000956) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000957) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000958) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000959) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000960) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000961) opened an unlinked file! 2024-04-01T01:43:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000962) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000963) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000964) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000965) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000966) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000967) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000968) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000969) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000970) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000971) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000972) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000973) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000974) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000975) opened an unlinked file! 2024-04-01T01:43:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000976) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000977) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000978) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000979) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000980) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000981) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000982) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000983) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000984) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000985) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000986) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000987) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000988) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000989) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000990) opened an unlinked file! 2024-04-01T01:43:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000991) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000992) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000993) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000994) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000995) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000996) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000997) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000998) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file000999) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001000) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001001) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001002) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001003) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001004) opened an unlinked file! 2024-04-01T01:43:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001005) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001006) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001007) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001008) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001009) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001010) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001011) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001012) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001013) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001014) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001015) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001016) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001017) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001018) opened an unlinked file! 2024-04-01T01:43:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001019) opened an unlinked file! 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001020) opened an unlinked file! 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001021) opened an unlinked file! 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001022) opened an unlinked file! 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/426-dir/file001023) opened an unlinked file! 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/426-dir 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/426-dir -l 2024-04-01T01:43:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/426-dir -u 2024-04-01T01:43:03.196 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/426 2024-04-01T01:43:03.196 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/426 2024-04-01T01:43:03.196 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:43:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:03.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:43:03.334 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:43:03.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:43:03.335 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-04-01T01:43:04.337 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:04.341 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:04.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:04.510 INFO:teuthology.orchestra.run.smithi064.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:43:04.513 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/427 2024-04-01T01:43:04.513 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/427 2024-04-01T01:43:04.513 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:04.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:04.523 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-04-01T01:43:04.698 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:04.701 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:04.702 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:04.893 INFO:teuthology.orchestra.run.smithi064.stdout:generic/428 0s 2024-04-01T01:43:04.908 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/428 2024-04-01T01:43:04.908 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:04.910 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:04.919 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:04.919 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-04-01T01:43:05.094 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:05.097 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:05.098 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:05.249 INFO:teuthology.orchestra.run.smithi064.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:43:05.253 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/429 2024-04-01T01:43:05.253 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/429 2024-04-01T01:43:05.253 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:05.255 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:05.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:05.263 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-04-01T01:43:05.435 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:05.439 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:05.439 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:05.717 INFO:teuthology.orchestra.run.smithi064.stdout:generic/430 0s 2024-04-01T01:43:05.723 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/430 2024-04-01T01:43:05.723 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:05.725 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:05.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:05.734 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-04-01T01:43:06.282 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:06.286 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:06.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:06.528 INFO:teuthology.orchestra.run.smithi064.stdout:generic/431 0s 2024-04-01T01:43:06.533 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/431 2024-04-01T01:43:06.533 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:06.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:06.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:06.543 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-04-01T01:43:06.952 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:06.956 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:06.956 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:07.219 INFO:teuthology.orchestra.run.smithi064.stdout:generic/432 1s 2024-04-01T01:43:07.224 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/432 2024-04-01T01:43:07.224 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:07.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:07.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:07.235 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-04-01T01:43:07.656 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:07.666 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:07.886 INFO:teuthology.orchestra.run.smithi064.stdout:generic/433 0s 2024-04-01T01:43:07.891 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/433 2024-04-01T01:43:07.892 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:07.894 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:07.902 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:07.902 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-04-01T01:43:08.377 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:08.381 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:08.381 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:08.603 INFO:teuthology.orchestra.run.smithi064.stdout:generic/434 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/434.out.bad) 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/434.out 2024-04-01 00:14:57.022907943 +0000 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/434.out.bad 2024-04-01 01:43:08.602689014 +0000 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -9,6 +9,6 @@ 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: copy_range: Bad file descriptor 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: Try to copy to a device 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: -copy_range: Invalid argument 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: +/tmp/tmp.vihoW6DmoUtest/test-434/dev1: Permission denied 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: Try to copy to a fifo 2024-04-01T01:43:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: copy_range: Invalid argument 2024-04-01T01:43:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/434 2024-04-01T01:43:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/434 2024-04-01T01:43:08.607 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:43:08.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:08.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:43:08.617 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:43:08.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:43:08.617 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-04-01T01:43:09.052 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:09.056 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:09.056 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:09.199 INFO:teuthology.orchestra.run.smithi064.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:43:09.203 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/435 2024-04-01T01:43:09.203 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/435 2024-04-01T01:43:09.203 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:09.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:09.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:09.213 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-04-01T01:43:09.386 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:09.390 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:09.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:10.536 INFO:teuthology.orchestra.run.smithi064.stdout:generic/436 1s 2024-04-01T01:43:10.542 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/436 2024-04-01T01:43:10.542 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:10.544 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:10.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:10.552 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-04-01T01:43:10.991 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:10.995 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:11.818 INFO:teuthology.orchestra.run.smithi064.stdout:generic/437 0s 2024-04-01T01:43:11.823 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/437 2024-04-01T01:43:11.823 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:43:11.825 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:11.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:43:11.834 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-04-01T01:43:12.013 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:43:12.017 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:43:12.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:43:40.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:43:40.461 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:44:10.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:44:10.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:44:11.226 INFO:teuthology.orchestra.run.smithi064.stdout:generic/438 59s 2024-04-01T01:44:11.231 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/438 2024-04-01T01:44:11.232 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:11.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:11.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:11.242 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-04-01T01:44:11.417 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:11.421 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:11.421 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:11.539 INFO:teuthology.orchestra.run.smithi064.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:11.542 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/439 2024-04-01T01:44:11.542 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/439 2024-04-01T01:44:11.542 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:11.544 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:11.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:11.553 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-04-01T01:44:11.727 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:11.731 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:11.731 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:11.889 INFO:teuthology.orchestra.run.smithi064.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:11.893 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/440 2024-04-01T01:44:11.893 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/440 2024-04-01T01:44:11.893 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:11.895 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:11.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:11.903 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-04-01T01:44:12.076 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:12.080 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:12.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.254 INFO:teuthology.orchestra.run.smithi064.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:12.257 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/441 2024-04-01T01:44:12.257 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/441 2024-04-01T01:44:12.257 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:12.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:12.268 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-04-01T01:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.568 INFO:teuthology.orchestra.run.smithi064.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:12.571 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/442 2024-04-01T01:44:12.571 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/442 2024-04-01T01:44:12.571 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:12.582 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-04-01T01:44:12.753 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:12.757 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:12.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.905 INFO:teuthology.orchestra.run.smithi064.stdout:generic/443 0s 2024-04-01T01:44:12.910 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/443 2024-04-01T01:44:12.910 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:12.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:12.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:12.921 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-04-01T01:44:13.106 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:13.109 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:13.109 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:13.275 INFO:teuthology.orchestra.run.smithi064.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T01:44:13.278 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/444 2024-04-01T01:44:13.279 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/444 2024-04-01T01:44:13.279 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:13.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:13.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:13.289 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-04-01T01:44:13.462 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:13.466 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:13.466 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:14.438 INFO:teuthology.orchestra.run.smithi064.stdout:generic/445 1s 2024-04-01T01:44:14.442 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/445 2024-04-01T01:44:14.443 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:14.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:14.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:14.453 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-04-01T01:44:14.871 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:14.876 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:14.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:14.995 INFO:teuthology.orchestra.run.smithi064.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:14.999 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/446 2024-04-01T01:44:14.999 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/446 2024-04-01T01:44:14.999 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:15.001 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:15.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:15.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-04-01T01:44:15.194 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:15.197 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:15.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:15.334 INFO:teuthology.orchestra.run.smithi064.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:15.337 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/447 2024-04-01T01:44:15.337 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/447 2024-04-01T01:44:15.337 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:15.340 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:15.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:15.348 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-04-01T01:44:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:15.535 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:16.515 INFO:teuthology.orchestra.run.smithi064.stdout:generic/448 1s 2024-04-01T01:44:16.520 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/448 2024-04-01T01:44:16.520 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:16.523 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:16.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:16.531 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-04-01T01:44:16.938 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:16.942 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:16.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:17.070 INFO:teuthology.orchestra.run.smithi064.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/449 2024-04-01T01:44:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/449 2024-04-01T01:44:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:17.083 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:17.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-04-01T01:44:17.255 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:17.259 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:18.141 INFO:teuthology.orchestra.run.smithi064.stdout:generic/450 1s 2024-04-01T01:44:18.146 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/450 2024-04-01T01:44:18.146 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:18.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:18.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:18.157 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-04-01T01:44:18.586 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:18.590 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:18.895 INFO:teuthology.orchestra.run.smithi064.stdout:generic/451 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/451.out.bad) 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/451.out 2024-04-01 00:14:57.023907922 +0000 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/451.out.bad 2024-04-01 01:44:18.894319202 +0000 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,2 +1,5 @@ 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 451 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: +get stale data from buffer read 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: +000a0000 2024-04-01T01:44:18.897 INFO:teuthology.orchestra.run.smithi064.stdout: Silence is golden 2024-04-01T01:44:18.899 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/451 2024-04-01T01:44:18.899 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/451 2024-04-01T01:44:18.899 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:44:18.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:18.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:44:18.910 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:44:18.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:44:18.910 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-04-01T01:44:19.268 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:19.275 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:19.444 INFO:teuthology.orchestra.run.smithi064.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:19.449 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/452 2024-04-01T01:44:19.450 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/452 2024-04-01T01:44:19.450 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:19.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:19.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:19.464 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-04-01T01:44:19.637 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:19.641 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:19.641 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:19.758 INFO:teuthology.orchestra.run.smithi064.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:19.761 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/453 2024-04-01T01:44:19.761 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/453 2024-04-01T01:44:19.761 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:19.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:19.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:19.771 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-04-01T01:44:19.954 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:19.957 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:19.957 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.066 INFO:teuthology.orchestra.run.smithi064.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:20.070 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/454 2024-04-01T01:44:20.070 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/454 2024-04-01T01:44:20.070 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:20.072 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:20.080 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-04-01T01:44:20.260 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:20.264 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:20.264 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.430 INFO:teuthology.orchestra.run.smithi064.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:20.433 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/455 2024-04-01T01:44:20.433 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/455 2024-04-01T01:44:20.433 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:20.436 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:20.456 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-04-01T01:44:20.628 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:20.631 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:20.631 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.757 INFO:teuthology.orchestra.run.smithi064.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:20.760 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/456 2024-04-01T01:44:20.760 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/456 2024-04-01T01:44:20.760 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:20.762 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:20.770 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:20.771 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-04-01T01:44:20.943 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:20.947 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:21.114 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/457 2024-04-01T01:44:21.114 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/457 2024-04-01T01:44:21.114 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:21.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:21.125 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-04-01T01:44:21.301 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:21.305 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.419 INFO:teuthology.orchestra.run.smithi064.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:21.422 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/458 2024-04-01T01:44:21.422 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/458 2024-04-01T01:44:21.422 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:21.425 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:21.433 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-04-01T01:44:21.601 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:21.606 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.837 INFO:teuthology.orchestra.run.smithi064.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:21.841 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/459 2024-04-01T01:44:21.841 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/459 2024-04-01T01:44:21.841 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:21.843 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:21.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:21.851 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-04-01T01:44:22.030 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:22.034 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.161 INFO:teuthology.orchestra.run.smithi064.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:22.165 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/460 2024-04-01T01:44:22.165 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/460 2024-04-01T01:44:22.165 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:22.167 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:22.175 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-04-01T01:44:22.350 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:22.354 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:22.354 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.475 INFO:teuthology.orchestra.run.smithi064.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:22.478 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/461 2024-04-01T01:44:22.479 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/461 2024-04-01T01:44:22.479 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:22.489 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-04-01T01:44:22.674 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:22.678 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.797 INFO:teuthology.orchestra.run.smithi064.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:22.800 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/462 2024-04-01T01:44:22.800 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/462 2024-04-01T01:44:22.801 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:22.803 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:22.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:22.811 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-04-01T01:44:22.985 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:22.988 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:22.988 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:23.158 INFO:teuthology.orchestra.run.smithi064.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T01:44:23.161 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/463 2024-04-01T01:44:23.161 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/463 2024-04-01T01:44:23.161 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:23.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:23.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:23.171 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-04-01T01:44:23.598 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:23.602 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:23.602 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:23.718 INFO:teuthology.orchestra.run.smithi064.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/464 2024-04-01T01:44:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/464 2024-04-01T01:44:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:23.724 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:23.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:23.732 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-04-01T01:44:23.907 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:23.911 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:23.911 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:38.186 INFO:teuthology.orchestra.run.smithi064.stdout:generic/465 15s 2024-04-01T01:44:38.195 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/465 2024-04-01T01:44:38.195 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:38.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:38.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:38.225 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-04-01T01:44:38.637 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:38.641 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:38.758 INFO:teuthology.orchestra.run.smithi064.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:38.762 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/466 2024-04-01T01:44:38.762 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/466 2024-04-01T01:44:38.762 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:38.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:38.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:38.772 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-04-01T01:44:38.946 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:38.951 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:38.952 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:40.634 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:44:40.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:44:44.473 INFO:teuthology.orchestra.run.smithi064.stdout:generic/467 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/467.out.bad) 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/467.out 2024-04-01 00:14:57.024907903 +0000 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/467.out.bad 2024-04-01 01:44:44.453821117 +0000 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,5 +1,15 @@ 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 467 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/467-dir -dp 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000000) opened an unlinked file! 2024-04-01T01:44:44.485 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000001) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000002) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000003) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000004) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000005) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000006) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000007) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000008) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: +open_by_handle(/tmp/tmp.vihoW6DmoUtest/467-dir/file000009) opened an unlinked file! 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/467-dir -rp 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/467-dir -dkr 2024-04-01T01:44:44.486 INFO:teuthology.orchestra.run.smithi064.stdout: test_file_handles TEST_DIR/467-dir -lr 2024-04-01T01:44:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/467 2024-04-01T01:44:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/467 2024-04-01T01:44:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:44:44.541 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:44.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:44:44.708 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:44:44.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:44:44.708 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-04-01T01:44:45.706 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:45.710 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:45.710 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:45.866 INFO:teuthology.orchestra.run.smithi064.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:45.905 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/468 2024-04-01T01:44:45.905 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/468 2024-04-01T01:44:45.905 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:45.905 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:45.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:45.906 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-04-01T01:44:46.077 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:46.081 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:46.081 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:47.609 INFO:teuthology.orchestra.run.smithi064.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-01T01:44:47.612 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/469 2024-04-01T01:44:47.612 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/469 2024-04-01T01:44:47.613 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:47.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:47.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:47.623 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-04-01T01:44:48.059 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:48.063 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:48.063 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:48.241 INFO:teuthology.orchestra.run.smithi064.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:44:48.244 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/470 2024-04-01T01:44:48.244 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/470 2024-04-01T01:44:48.244 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:44:48.246 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:44:48.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:44:48.255 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-04-01T01:44:48.437 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:44:48.441 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:44:48.441 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:12.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:45:12.781 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:45:12.812 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T01:45:12.812+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47343) UID: 0 2024-04-01T01:45:12.812 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T01:45:12.812+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47343) UID: 0 2024-04-01T01:45:52.231 INFO:teuthology.orchestra.run.smithi064.stdout:generic/471 64s 2024-04-01T01:45:52.236 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/471 2024-04-01T01:45:52.236 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:52.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:52.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:52.248 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-04-01T01:45:52.684 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:52.690 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:52.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:52.940 INFO:teuthology.orchestra.run.smithi064.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:45:52.945 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/472 2024-04-01T01:45:52.945 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/472 2024-04-01T01:45:52.945 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:52.948 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:52.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:52.965 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-04-01T01:45:53.301 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:53.305 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:53.306 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:53.506 INFO:teuthology.orchestra.run.smithi064.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T01:45:53.509 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/473 2024-04-01T01:45:53.509 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/473 2024-04-01T01:45:53.509 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:53.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:53.520 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-04-01T01:45:53.691 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:53.694 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:53.695 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:53.810 INFO:teuthology.orchestra.run.smithi064.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:45:53.813 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/474 2024-04-01T01:45:53.813 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/474 2024-04-01T01:45:53.813 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:53.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:53.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:53.824 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-04-01T01:45:54.010 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:54.014 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:54.014 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:54.426 INFO:teuthology.orchestra.run.smithi064.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:45:54.429 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/475 2024-04-01T01:45:54.430 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/475 2024-04-01T01:45:54.430 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:54.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:54.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-04-01T01:45:54.622 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:54.626 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:54.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:55.088 INFO:teuthology.orchestra.run.smithi064.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:45:55.091 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/476 2024-04-01T01:45:55.091 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/476 2024-04-01T01:45:55.092 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:45:55.093 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:45:55.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:45:55.103 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-04-01T01:45:55.277 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:45:55.281 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:45:55.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:01.492 INFO:teuthology.orchestra.run.smithi064.stdout:generic/477 6s 2024-04-01T01:46:01.586 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/477 2024-04-01T01:46:01.587 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:01.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:01.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:01.668 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-04-01T01:46:02.611 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:02.616 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:02.616 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:12.638 INFO:teuthology.orchestra.run.smithi064.stdout:generic/478 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/478.out.bad) 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/478.out 2024-04-01 00:14:57.025907883 +0000 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/478.out.bad 2024-04-01 01:46:12.626103058 +0000 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -10,7 +10,7 @@ 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: get wrlck 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: -lock could be placed 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: +get wrlck 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -25,7 +25,7 @@ 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: get wrlck 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: -lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: +get wrlck 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -37,7 +37,7 @@ 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: -lock could be placed 2024-04-01T01:46:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: +get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -46,7 +46,7 @@ 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: -lock could be placed 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: +get rdlck 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: lock could be placed 2024-04-01T01:46:12.643 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/478 2024-04-01T01:46:12.643 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/478 2024-04-01T01:46:12.643 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:46:12.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:12.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:46:12.656 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:46:12.656 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:46:12.656 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-04-01T01:46:12.844 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:12.848 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:12.848 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.013 INFO:teuthology.orchestra.run.smithi064.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:13.016 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/479 2024-04-01T01:46:13.016 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/479 2024-04-01T01:46:13.016 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:13.018 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:13.041 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-04-01T01:46:13.225 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:13.229 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:13.229 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.358 INFO:teuthology.orchestra.run.smithi064.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:13.362 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/480 2024-04-01T01:46:13.362 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/480 2024-04-01T01:46:13.362 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:13.364 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:13.372 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-04-01T01:46:13.545 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:13.549 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.675 INFO:teuthology.orchestra.run.smithi064.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:13.679 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/481 2024-04-01T01:46:13.679 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/481 2024-04-01T01:46:13.679 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:13.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:13.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:13.690 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-04-01T01:46:13.867 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:13.871 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:13.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.050 INFO:teuthology.orchestra.run.smithi064.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:14.054 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/482 2024-04-01T01:46:14.054 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/482 2024-04-01T01:46:14.054 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:14.064 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-04-01T01:46:14.247 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:14.250 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:14.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.374 INFO:teuthology.orchestra.run.smithi064.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:14.377 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/483 2024-04-01T01:46:14.378 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/483 2024-04-01T01:46:14.378 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:14.388 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-04-01T01:46:14.559 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:14.562 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:14.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.705 INFO:teuthology.orchestra.run.smithi064.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:14.708 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/484 2024-04-01T01:46:14.708 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/484 2024-04-01T01:46:14.709 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:14.711 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:14.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:14.719 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-04-01T01:46:14.889 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:14.893 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:14.893 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:15.765 INFO:teuthology.orchestra.run.smithi064.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-01T01:46:15.768 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/485 2024-04-01T01:46:15.768 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/485 2024-04-01T01:46:15.768 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:15.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:15.779 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:15.779 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-04-01T01:46:16.176 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:16.181 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:16.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.330 INFO:teuthology.orchestra.run.smithi064.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:16.333 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/486 2024-04-01T01:46:16.334 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/486 2024-04-01T01:46:16.334 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:16.336 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:16.344 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-04-01T01:46:16.522 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:16.526 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:16.526 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.655 INFO:teuthology.orchestra.run.smithi064.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:16.658 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/487 2024-04-01T01:46:16.658 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/487 2024-04-01T01:46:16.659 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:16.669 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-04-01T01:46:16.840 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:16.844 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:16.844 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.957 INFO:teuthology.orchestra.run.smithi064.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:16.960 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/488 2024-04-01T01:46:16.960 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/488 2024-04-01T01:46:16.960 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:16.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:16.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:16.970 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-04-01T01:46:17.152 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:17.156 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:17.156 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:17.283 INFO:teuthology.orchestra.run.smithi064.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:17.286 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/489 2024-04-01T01:46:17.286 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/489 2024-04-01T01:46:17.286 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:17.288 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:17.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:17.297 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-04-01T01:46:17.470 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:17.474 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:17.474 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:18.410 INFO:teuthology.orchestra.run.smithi064.stdout:generic/490 1s 2024-04-01T01:46:18.415 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/490 2024-04-01T01:46:18.416 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:18.418 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:18.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:18.427 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-04-01T01:46:18.878 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:18.883 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:18.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.011 INFO:teuthology.orchestra.run.smithi064.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/491 2024-04-01T01:46:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/491 2024-04-01T01:46:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:19.026 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-04-01T01:46:19.200 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:19.204 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:19.204 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:19.326 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/492 2024-04-01T01:46:19.327 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/492 2024-04-01T01:46:19.327 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:19.329 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:19.338 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-04-01T01:46:19.515 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:19.519 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:19.519 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.638 INFO:teuthology.orchestra.run.smithi064.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:19.642 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/493 2024-04-01T01:46:19.642 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/493 2024-04-01T01:46:19.642 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:19.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:19.663 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-04-01T01:46:19.840 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:19.844 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:19.844 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.961 INFO:teuthology.orchestra.run.smithi064.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:19.964 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/494 2024-04-01T01:46:19.964 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/494 2024-04-01T01:46:19.964 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:19.966 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:19.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:19.975 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-04-01T01:46:20.158 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:20.162 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:20.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.277 INFO:teuthology.orchestra.run.smithi064.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:20.281 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/495 2024-04-01T01:46:20.281 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/495 2024-04-01T01:46:20.281 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:20.283 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:20.291 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-04-01T01:46:20.476 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:20.480 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:20.480 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.603 INFO:teuthology.orchestra.run.smithi064.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:20.606 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/496 2024-04-01T01:46:20.606 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/496 2024-04-01T01:46:20.606 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:20.608 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:20.616 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-04-01T01:46:20.790 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:20.794 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:20.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.914 INFO:teuthology.orchestra.run.smithi064.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:20.917 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/497 2024-04-01T01:46:20.917 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/497 2024-04-01T01:46:20.917 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:20.919 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:20.928 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:20.928 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-04-01T01:46:21.113 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:21.118 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:21.118 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.247 INFO:teuthology.orchestra.run.smithi064.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/498 2024-04-01T01:46:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/498 2024-04-01T01:46:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:21.253 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:21.262 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-04-01T01:46:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:21.444 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:21.444 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:21.565 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/499 2024-04-01T01:46:21.565 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/499 2024-04-01T01:46:21.565 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:21.576 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-04-01T01:46:21.757 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:21.761 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:21.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.922 INFO:teuthology.orchestra.run.smithi064.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:21.925 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/500 2024-04-01T01:46:21.926 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/500 2024-04-01T01:46:21.926 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:21.928 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:21.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:21.936 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-04-01T01:46:22.115 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.251 INFO:teuthology.orchestra.run.smithi064.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:22.255 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/501 2024-04-01T01:46:22.255 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/501 2024-04-01T01:46:22.255 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:22.257 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:22.285 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-04-01T01:46:22.463 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:22.467 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.599 INFO:teuthology.orchestra.run.smithi064.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:22.602 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/502 2024-04-01T01:46:22.602 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/502 2024-04-01T01:46:22.603 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:22.613 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-04-01T01:46:22.795 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:22.798 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:22.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.916 INFO:teuthology.orchestra.run.smithi064.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:22.920 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/503 2024-04-01T01:46:22.920 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/503 2024-04-01T01:46:22.920 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:22.922 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:22.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:22.931 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-04-01T01:46:23.113 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:23.117 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:23.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.281 INFO:teuthology.orchestra.run.smithi064.stdout:generic/504 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/504.out.bad) 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/504.out 2024-04-01 00:14:57.027907843 +0000 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/504.out.bad 2024-04-01 01:46:23.279895467 +0000 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,2 +1,3 @@ 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 504 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: +lock info not found 2024-04-01T01:46:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: Silence is golden 2024-04-01T01:46:23.285 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/504 2024-04-01T01:46:23.285 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/504 2024-04-01T01:46:23.285 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T01:46:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T01:46:23.296 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T01:46:23.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T01:46:23.297 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-04-01T01:46:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:23.475 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:23.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.599 INFO:teuthology.orchestra.run.smithi064.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:23.602 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/505 2024-04-01T01:46:23.602 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/505 2024-04-01T01:46:23.602 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:23.604 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:23.613 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-04-01T01:46:23.790 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:23.794 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:23.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.906 INFO:teuthology.orchestra.run.smithi064.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:23.910 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/506 2024-04-01T01:46:23.910 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/506 2024-04-01T01:46:23.910 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:23.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:23.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:23.921 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-04-01T01:46:24.091 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:24.095 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:24.095 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.280 INFO:teuthology.orchestra.run.smithi064.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-04-01T01:46:24.284 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/507 2024-04-01T01:46:24.284 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/507 2024-04-01T01:46:24.284 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:24.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:24.305 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-04-01T01:46:24.477 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:24.493 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:24.493 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.611 INFO:teuthology.orchestra.run.smithi064.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2024-04-01T01:46:24.614 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/508 2024-04-01T01:46:24.614 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/508 2024-04-01T01:46:24.615 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:24.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:24.625 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-04-01T01:46:24.810 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:24.814 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:24.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.937 INFO:teuthology.orchestra.run.smithi064.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:24.940 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/509 2024-04-01T01:46:24.940 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/509 2024-04-01T01:46:24.940 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:24.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:24.951 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-04-01T01:46:25.122 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:25.126 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:25.126 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.261 INFO:teuthology.orchestra.run.smithi064.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:25.264 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/510 2024-04-01T01:46:25.264 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/510 2024-04-01T01:46:25.265 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:25.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:25.282 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-04-01T01:46:25.457 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:25.461 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:25.461 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.579 INFO:teuthology.orchestra.run.smithi064.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:25.582 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/511 2024-04-01T01:46:25.582 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/511 2024-04-01T01:46:25.582 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:25.584 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:25.592 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-04-01T01:46:25.767 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:25.771 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:25.771 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.901 INFO:teuthology.orchestra.run.smithi064.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:25.904 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/512 2024-04-01T01:46:25.904 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/512 2024-04-01T01:46:25.904 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:25.906 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:25.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:25.915 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-04-01T01:46:26.088 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:26.092 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:26.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.216 INFO:teuthology.orchestra.run.smithi064.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:26.220 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/513 2024-04-01T01:46:26.220 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/513 2024-04-01T01:46:26.220 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:26.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:26.231 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-04-01T01:46:26.408 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:26.412 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:26.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.526 INFO:teuthology.orchestra.run.smithi064.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:26.530 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/514 2024-04-01T01:46:26.530 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/514 2024-04-01T01:46:26.530 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:26.532 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:26.541 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-04-01T01:46:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:26.719 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.837 INFO:teuthology.orchestra.run.smithi064.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:26.841 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/515 2024-04-01T01:46:26.841 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/515 2024-04-01T01:46:26.841 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:26.843 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:26.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:26.859 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-04-01T01:46:27.038 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:27.042 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:27.202 INFO:teuthology.orchestra.run.smithi064.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-01T01:46:27.205 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/516 2024-04-01T01:46:27.206 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/516 2024-04-01T01:46:27.206 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:27.208 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:27.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:27.217 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-04-01T01:46:27.617 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:27.622 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:27.759 INFO:teuthology.orchestra.run.smithi064.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:27.763 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/517 2024-04-01T01:46:27.763 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/517 2024-04-01T01:46:27.763 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:27.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:27.773 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-04-01T01:46:27.951 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:27.955 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.069 INFO:teuthology.orchestra.run.smithi064.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:28.073 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/518 2024-04-01T01:46:28.073 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/518 2024-04-01T01:46:28.073 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.083 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:28.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-04-01T01:46:28.257 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:28.261 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.372 INFO:teuthology.orchestra.run.smithi064.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/519 2024-04-01T01:46:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/519 2024-04-01T01:46:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:28.395 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-04-01T01:46:28.565 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:28.569 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.698 INFO:teuthology.orchestra.run.smithi064.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T01:46:28.701 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/520 2024-04-01T01:46:28.702 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/520 2024-04-01T01:46:28.702 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T01:46:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:46:28.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T01:46:28.712 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-04-01T01:46:28.886 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T01:46:28.890 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T01:46:28.890 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T01:48:12.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:48:12.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:48:42.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:48:42.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:49:12.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:49:12.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:49:42.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:49:42.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:50:12.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:50:12.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:50:42.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:50:42.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:51:12.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:51:12.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:51:42.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:51:42.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:52:13.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:52:13.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:52:43.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:52:43.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:53:13.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:53:13.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:53:43.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:53:43.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:54:13.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:54:13.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:54:43.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:54:43.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:55:13.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:55:13.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:55:43.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:55:43.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:56:14.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:56:14.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:56:44.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:56:44.074 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:57:14.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:57:14.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:57:44.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:57:44.393 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:58:14.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:58:14.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:58:44.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:58:44.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:59:15.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:59:15.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:59:45.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T01:59:45.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:00:15.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:00:15.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:00:45.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:00:45.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:01:15.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:01:15.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:01:45.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:01:45.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:02:15.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:02:15.543 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:02:45.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:02:45.607 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:03:15.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:03:15.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:03:45.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:03:45.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:04:15.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:04:15.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:04:45.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:04:45.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:05:16.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:05:16.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:05:46.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:05:46.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:06:16.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:06:16.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:06:46.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:06:46.308 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:07:16.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:07:16.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:07:16.471 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T02:07:16.471+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 444720) UID: 0 2024-04-01T02:07:16.472 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T02:07:16.471+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 444720) UID: 0 2024-04-01T02:07:16.472 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T02:07:16.471+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 444720) UID: 0 2024-04-01T02:07:16.472 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T02:07:16.471+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 444720) UID: 0 2024-04-01T02:09:38.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:09:38.043 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:10:08.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:10:08.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:10:38.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:10:38.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:11:08.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:11:08.447 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:11:38.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:11:38.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:12:08.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:12:08.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:12:38.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:12:38.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:13:08.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:13:08.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:13:38.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:13:38.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:14:09.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:14:09.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:14:39.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:14:39.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:15:09.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:15:09.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:15:39.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:15:39.311 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:16:09.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:16:09.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:16:39.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:16:39.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:17:09.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:17:09.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:17:39.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:17:39.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:18:09.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:18:09.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:18:39.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:18:39.844 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:19:09.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:19:09.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:19:40.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:19:40.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:20:10.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:20:10.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:20:40.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:20:40.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:21:10.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:21:10.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:21:40.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:21:40.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:22:10.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:22:10.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:22:40.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:22:40.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:23:10.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:23:10.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:23:40.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:23:40.913 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:24:10.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:24:10.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:24:41.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:24:41.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:25:11.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:25:11.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:25:41.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:25:41.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:26:11.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:26:11.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:26:41.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:26:41.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:27:11.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:27:11.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:27:42.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:27:42.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:28:12.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:28:12.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:28:14.737 INFO:teuthology.orchestra.run.smithi064.stdout:generic/521 2506s 2024-04-01T02:28:14.742 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/521 2024-04-01T02:28:14.742 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T02:28:14.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T02:28:14.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T02:28:14.752 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-04-01T02:28:15.236 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T02:28:15.240 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T02:28:15.240 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T02:28:42.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:28:42.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:29:12.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:29:12.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:29:42.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:29:42.536 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:30:12.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:30:12.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:30:42.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:30:42.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:31:12.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:31:12.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:31:43.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:31:43.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:32:13.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:32:13.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:32:43.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:32:43.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:33:13.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:33:13.383 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:33:43.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:33:43.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:34:13.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:34:13.895 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:34:44.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:34:44.152 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:34:44.184 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T02:34:44.183+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 49026) UID: 0 2024-04-01T02:34:44.184 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T02:34:44.183+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 49026) UID: 0 2024-04-01T02:37:24.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:37:24.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:37:54.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:37:54.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:38:24.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:38:24.447 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:38:54.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:38:54.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:39:24.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:39:24.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:39:54.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:39:54.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:40:25.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:40:25.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:40:55.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:40:55.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:41:25.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:41:25.455 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:41:55.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:41:55.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:42:25.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:42:25.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:42:55.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:42:55.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:43:25.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:43:25.884 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:43:56.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:43:56.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:44:26.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:44:26.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:44:26.137 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T02:44:26.137+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49306) UID: 0 2024-04-01T02:44:26.137 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T02:44:26.137+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49306) UID: 0 2024-04-01T02:44:26.137 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T02:44:26.137+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49306) UID: 0 2024-04-01T02:44:26.138 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T02:44:26.137+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49306) UID: 0 2024-04-01T02:46:48.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:46:48.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:47:19.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:47:19.158 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:47:49.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:47:49.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:48:19.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:48:19.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:48:49.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:48:49.452 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:49:19.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:49:19.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:49:49.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:49:49.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:50:19.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:50:19.806 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:50:49.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:50:49.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:51:20.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:51:20.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:51:50.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:51:50.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:52:20.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:52:20.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:52:50.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:52:50.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:53:20.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:53:20.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:53:50.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:53:50.742 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:54:20.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:54:20.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:54:50.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:54:50.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:55:21.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:55:21.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:55:51.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:55:51.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:56:21.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:56:21.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:56:51.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:56:51.497 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:57:21.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:57:21.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:57:51.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:57:51.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:58:21.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:58:21.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:58:52.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:58:52.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:59:22.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:59:22.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:59:52.315 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T02:59:52.319 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:00:22.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:00:22.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:00:52.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:00:52.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:01:22.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:01:22.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:01:52.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:01:52.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:02:12.201 INFO:teuthology.orchestra.run.smithi064.stdout:generic/522 2037s 2024-04-01T03:02:12.223 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/522 2024-04-01T03:02:12.224 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:12.225 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:12.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:12.235 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-04-01T03:02:12.525 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:12.529 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:12.529 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:12.671 INFO:teuthology.orchestra.run.smithi064.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:12.675 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/523 2024-04-01T03:02:12.675 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/523 2024-04-01T03:02:12.675 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:12.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:12.686 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-04-01T03:02:12.861 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:12.866 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:12.866 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:12.982 INFO:teuthology.orchestra.run.smithi064.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:12.985 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/524 2024-04-01T03:02:12.985 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/524 2024-04-01T03:02:12.985 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:12.987 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:12.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:12.995 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-04-01T03:02:13.169 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:13.172 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:13.303 INFO:teuthology.orchestra.run.smithi064.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:13.307 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/525 2024-04-01T03:02:13.307 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/525 2024-04-01T03:02:13.307 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:13.309 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:13.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:13.318 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-04-01T03:02:13.495 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:13.499 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:13.499 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:13.692 INFO:teuthology.orchestra.run.smithi064.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/526 2024-04-01T03:02:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/526 2024-04-01T03:02:13.696 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:13.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:13.706 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-04-01T03:02:13.879 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:13.882 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:13.882 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.009 INFO:teuthology.orchestra.run.smithi064.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:14.013 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/527 2024-04-01T03:02:14.013 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/527 2024-04-01T03:02:14.013 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:14.015 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:14.023 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-04-01T03:02:14.199 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:14.203 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:14.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.343 INFO:teuthology.orchestra.run.smithi064.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2024-04-01T03:02:14.346 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/528 2024-04-01T03:02:14.346 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/528 2024-04-01T03:02:14.346 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:14.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-04-01T03:02:14.535 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:14.539 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:14.539 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.711 INFO:teuthology.orchestra.run.smithi064.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T03:02:14.714 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/529 2024-04-01T03:02:14.714 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/529 2024-04-01T03:02:14.715 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:14.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:14.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:14.724 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-04-01T03:02:14.906 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:14.910 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:14.910 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.022 INFO:teuthology.orchestra.run.smithi064.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:15.026 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/530 2024-04-01T03:02:15.026 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/530 2024-04-01T03:02:15.026 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:15.028 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:15.036 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-04-01T03:02:15.213 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:15.216 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.335 INFO:teuthology.orchestra.run.smithi064.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:15.339 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/531 2024-04-01T03:02:15.339 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/531 2024-04-01T03:02:15.339 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:15.349 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-04-01T03:02:15.527 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:15.531 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:generic/532 0s 2024-04-01T03:02:15.707 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/532 2024-04-01T03:02:15.707 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:15.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:15.717 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:15.717 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-04-01T03:02:15.893 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:15.897 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:15.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.263 INFO:teuthology.orchestra.run.smithi064.stdout:generic/533 1s 2024-04-01T03:02:16.268 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/533 2024-04-01T03:02:16.268 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:16.270 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:16.278 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-04-01T03:02:16.457 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:16.461 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:16.461 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.584 INFO:teuthology.orchestra.run.smithi064.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:16.588 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/534 2024-04-01T03:02:16.588 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/534 2024-04-01T03:02:16.588 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:16.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:16.598 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-04-01T03:02:16.774 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:16.778 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:16.778 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.921 INFO:teuthology.orchestra.run.smithi064.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:16.925 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/535 2024-04-01T03:02:16.925 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/535 2024-04-01T03:02:16.925 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:16.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:16.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:16.935 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-04-01T03:02:17.103 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:17.107 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:17.228 INFO:teuthology.orchestra.run.smithi064.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:17.231 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/536 2024-04-01T03:02:17.231 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/536 2024-04-01T03:02:17.232 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:17.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:17.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:17.242 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-04-01T03:02:17.416 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:17.419 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:17.420 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:17.544 INFO:teuthology.orchestra.run.smithi064.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:02:17.548 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/537 2024-04-01T03:02:17.548 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/537 2024-04-01T03:02:17.548 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:02:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:17.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:02:17.559 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-04-01T03:02:17.735 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:02:17.738 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:02:17.739 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:02:22.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:02:22.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:02:53.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:02:53.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:03:23.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:03:23.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:03:53.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:03:53.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:04:23.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:04:23.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:04:53.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:04:53.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:05:23.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:05:23.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:05:53.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:05:53.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:06:23.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:06:23.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:06:53.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:06:53.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:07:23.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:07:23.891 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:07:53.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:07:53.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:08:24.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:08:24.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:08:54.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:08:54.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:09:24.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:09:24.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:09:54.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:09:54.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:10:24.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:10:24.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:10:54.959 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:10:54.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:11:25.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:11:25.041 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:11:55.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:11:55.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:12:25.644 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:12:25.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:12:55.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:12:55.733 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:13:25.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:13:25.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:13:55.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:13:55.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:14:26.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:14:26.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:14:56.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:14:56.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:15:26.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:15:26.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:15:56.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:15:56.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:16:26.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:16:26.561 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:16:56.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:16:56.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:17:26.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:17:26.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:17:56.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:17:56.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:18:27.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:18:27.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:18:57.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:18:57.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:19:27.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:19:27.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:19:57.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:19:57.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:20:27.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:20:27.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:20:27.750 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T03:20:27.750+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 457612) UID: 0 2024-04-01T03:20:27.750 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T03:20:27.750+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 457612) UID: 0 2024-04-01T03:20:27.750 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T03:20:27.750+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 457612) UID: 0 2024-04-01T03:20:27.751 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T03:20:27.750+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 457612) UID: 0 2024-04-01T03:22:58.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:22:58.882 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:22:58.916 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T03:22:58.916+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 458017) UID: 0 2024-04-01T03:22:58.916 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T03:22:58.916+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 458017) UID: 0 2024-04-01T03:22:58.916 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T03:22:58.916+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 458017) UID: 0 2024-04-01T03:22:58.917 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T03:22:58.916+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 458017) UID: 0 2024-04-01T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:25:23.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:25:53.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:25:53.183 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:26:23.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:26:23.239 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:26:53.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:26:53.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:27:23.455 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:27:23.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:27:53.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:27:53.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:28:23.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:28:23.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:28:53.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:28:53.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:28:53.822 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T03:28:53.821+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 459066) UID: 0 2024-04-01T03:28:53.822 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T03:28:53.821+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 459066) UID: 0 2024-04-01T03:28:53.822 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T03:28:53.821+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 459066) UID: 0 2024-04-01T03:28:53.823 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T03:28:53.821+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 459066) UID: 0 2024-04-01T03:31:22.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:31:22.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:31:22.393 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T03:31:22.392+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50767) UID: 0 2024-04-01T03:31:22.393 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T03:31:22.392+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50767) UID: 0 2024-04-01T03:31:22.394 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T03:31:22.392+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50767) UID: 0 2024-04-01T03:31:22.394 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T03:31:22.392+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50767) UID: 0 2024-04-01T03:33:47.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:33:47.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:34:17.125 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:34:17.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:34:47.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:34:47.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:35:17.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:35:17.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:35:47.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:35:47.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:36:17.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:36:17.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:36:47.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:36:47.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:36:47.755 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T03:36:47.753+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50899) UID: 0 2024-04-01T03:36:47.755 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T03:36:47.753+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50899) UID: 0 2024-04-01T03:36:47.755 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T03:36:47.753+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50899) UID: 0 2024-04-01T03:36:47.755 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T03:36:47.753+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50899) UID: 0 2024-04-01T03:38:55.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:38:55.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:39:25.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:39:25.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:39:55.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:39:55.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:40:25.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:40:25.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:40:55.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:40:55.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:41:25.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:41:25.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:41:55.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:41:55.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:42:13.193 INFO:teuthology.orchestra.run.smithi064.stdout:generic/538 2396s 2024-04-01T03:42:13.197 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/538 2024-04-01T03:42:13.198 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:13.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:13.209 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-04-01T03:42:13.692 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:13.695 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:13.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:15.371 INFO:teuthology.orchestra.run.smithi064.stdout:generic/539 2s 2024-04-01T03:42:15.376 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/539 2024-04-01T03:42:15.376 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:15.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:15.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:15.387 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-04-01T03:42:15.875 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:15.879 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:15.879 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.003 INFO:teuthology.orchestra.run.smithi064.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/540 2024-04-01T03:42:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/540 2024-04-01T03:42:16.007 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:16.017 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-04-01T03:42:16.201 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:16.205 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:16.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.318 INFO:teuthology.orchestra.run.smithi064.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:16.322 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/541 2024-04-01T03:42:16.322 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/541 2024-04-01T03:42:16.322 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:16.324 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:16.332 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-04-01T03:42:16.505 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:16.509 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:16.509 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.628 INFO:teuthology.orchestra.run.smithi064.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:16.631 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/542 2024-04-01T03:42:16.632 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/542 2024-04-01T03:42:16.632 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:16.633 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.641 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:16.642 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-04-01T03:42:16.816 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:16.820 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:16.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:16.944 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/543 2024-04-01T03:42:16.944 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/543 2024-04-01T03:42:16.944 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:16.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:16.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:16.955 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-04-01T03:42:17.134 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:17.138 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:17.138 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:17.266 INFO:teuthology.orchestra.run.smithi064.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:17.270 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/544 2024-04-01T03:42:17.270 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/544 2024-04-01T03:42:17.270 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:17.272 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:17.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:17.280 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-04-01T03:42:17.452 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:17.456 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:17.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:17.635 INFO:teuthology.orchestra.run.smithi064.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-04-01T03:42:17.639 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/545 2024-04-01T03:42:17.639 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/545 2024-04-01T03:42:17.639 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:17.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:17.649 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-04-01T03:42:17.827 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:17.831 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:17.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.016 INFO:teuthology.orchestra.run.smithi064.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:18.020 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/546 2024-04-01T03:42:18.020 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/546 2024-04-01T03:42:18.020 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:18.022 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:18.030 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-04-01T03:42:18.209 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:18.213 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:18.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.345 INFO:teuthology.orchestra.run.smithi064.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:18.348 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/547 2024-04-01T03:42:18.348 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/547 2024-04-01T03:42:18.348 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:18.351 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:18.359 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-04-01T03:42:18.540 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:18.544 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.695 INFO:teuthology.orchestra.run.smithi064.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:18.698 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/548 2024-04-01T03:42:18.698 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/548 2024-04-01T03:42:18.698 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:18.700 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:18.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:18.709 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-04-01T03:42:18.885 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:18.889 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:18.889 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.035 INFO:teuthology.orchestra.run.smithi064.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:19.039 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/549 2024-04-01T03:42:19.039 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/549 2024-04-01T03:42:19.039 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:19.049 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-04-01T03:42:19.223 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:19.227 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:19.227 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.373 INFO:teuthology.orchestra.run.smithi064.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/550 2024-04-01T03:42:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/550 2024-04-01T03:42:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:19.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:19.387 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-04-01T03:42:19.558 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:19.563 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:19.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.683 INFO:teuthology.orchestra.run.smithi064.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:19.686 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/551 2024-04-01T03:42:19.686 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/551 2024-04-01T03:42:19.686 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:19.688 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:19.696 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:19.697 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2024-04-01T03:42:19.870 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:19.875 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:19.875 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.005 INFO:teuthology.orchestra.run.smithi064.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:20.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/552 2024-04-01T03:42:20.009 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/552 2024-04-01T03:42:20.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:20.011 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:20.019 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2024-04-01T03:42:20.195 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:20.199 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:20.199 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.374 INFO:teuthology.orchestra.run.smithi064.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2024-04-01T03:42:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/553 2024-04-01T03:42:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/553 2024-04-01T03:42:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:20.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:20.388 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2024-04-01T03:42:20.852 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:20.856 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:20.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.978 INFO:teuthology.orchestra.run.smithi064.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:20.981 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/554 2024-04-01T03:42:20.982 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/554 2024-04-01T03:42:20.982 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:20.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:20.993 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2024-04-01T03:42:21.165 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:21.169 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:21.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:21.327 INFO:teuthology.orchestra.run.smithi064.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2024-04-01T03:42:21.331 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/555 2024-04-01T03:42:21.331 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/555 2024-04-01T03:42:21.331 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:21.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:21.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:21.382 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2024-04-01T03:42:21.808 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:21.812 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:21.945 INFO:teuthology.orchestra.run.smithi064.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:21.949 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/556 2024-04-01T03:42:21.949 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/556 2024-04-01T03:42:21.949 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:21.951 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:21.959 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:21.960 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2024-04-01T03:42:22.135 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:22.139 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:22.139 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.275 INFO:teuthology.orchestra.run.smithi064.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:22.278 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/557 2024-04-01T03:42:22.278 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/557 2024-04-01T03:42:22.278 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:22.298 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2024-04-01T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.616 INFO:teuthology.orchestra.run.smithi064.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:22.620 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/558 2024-04-01T03:42:22.620 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/558 2024-04-01T03:42:22.620 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:22.630 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2024-04-01T03:42:22.811 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:22.815 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:22.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.940 INFO:teuthology.orchestra.run.smithi064.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:22.943 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/559 2024-04-01T03:42:22.943 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/559 2024-04-01T03:42:22.944 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:22.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:22.954 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2024-04-01T03:42:23.134 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:23.138 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:23.138 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.262 INFO:teuthology.orchestra.run.smithi064.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:23.265 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/560 2024-04-01T03:42:23.265 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/560 2024-04-01T03:42:23.265 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:23.276 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2024-04-01T03:42:23.452 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:23.456 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:23.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.569 INFO:teuthology.orchestra.run.smithi064.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:23.573 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/561 2024-04-01T03:42:23.573 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/561 2024-04-01T03:42:23.573 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:23.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:23.583 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2024-04-01T03:42:23.756 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:23.760 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:23.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.879 INFO:teuthology.orchestra.run.smithi064.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:23.882 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/562 2024-04-01T03:42:23.882 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/562 2024-04-01T03:42:23.882 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:23.884 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:23.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:23.907 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2024-04-01T03:42:24.083 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:24.087 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:24.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:24.238 INFO:teuthology.orchestra.run.smithi064.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:24.241 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/563 2024-04-01T03:42:24.242 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/563 2024-04-01T03:42:24.242 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:24.244 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:24.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:24.252 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2024-04-01T03:42:24.432 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:24.436 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:24.436 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:24.849 INFO:teuthology.orchestra.run.smithi064.stdout:generic/564 0s 2024-04-01T03:42:24.854 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/564 2024-04-01T03:42:24.854 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:24.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:24.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:24.864 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2024-04-01T03:42:25.340 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:25.344 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:25.344 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:25.464 INFO:teuthology.orchestra.run.smithi064.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:25.468 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/565 2024-04-01T03:42:25.468 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/565 2024-04-01T03:42:25.468 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:25.470 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:25.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:25.479 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2024-04-01T03:42:25.654 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:25.658 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:25.779 INFO:teuthology.orchestra.run.smithi064.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:25.782 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/566 2024-04-01T03:42:25.782 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/566 2024-04-01T03:42:25.782 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:25.785 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:25.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:25.793 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2024-04-01T03:42:25.970 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:25.973 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:25.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:25.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:42:25.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:42:26.096 INFO:teuthology.orchestra.run.smithi064.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:26.099 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/567 2024-04-01T03:42:26.099 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/567 2024-04-01T03:42:26.100 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:26.102 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:26.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:26.110 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2024-04-01T03:42:26.331 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:26.335 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:26.518 INFO:teuthology.orchestra.run.smithi064.stdout:generic/568 0s 2024-04-01T03:42:26.551 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/568 2024-04-01T03:42:26.551 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:26.554 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:26.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:26.568 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2024-04-01T03:42:27.039 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:27.043 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:27.209 INFO:teuthology.orchestra.run.smithi064.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:27.212 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/569 2024-04-01T03:42:27.212 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/569 2024-04-01T03:42:27.212 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:27.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:27.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:27.223 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2024-04-01T03:42:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/570 2024-04-01T03:42:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/570 2024-04-01T03:42:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:27.555 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:27.556 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2024-04-01T03:42:27.737 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:27.741 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:27.741 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:33.998 INFO:teuthology.orchestra.run.smithi064.stdout:generic/571 6s 2024-04-01T03:42:34.004 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/571 2024-04-01T03:42:34.005 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:34.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:34.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:34.017 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2024-04-01T03:42:34.482 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:34.486 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:34.487 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:34.606 INFO:teuthology.orchestra.run.smithi064.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:34.609 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/572 2024-04-01T03:42:34.610 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/572 2024-04-01T03:42:34.610 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:34.612 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:34.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:34.620 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2024-04-01T03:42:34.800 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:34.804 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:34.920 INFO:teuthology.orchestra.run.smithi064.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:34.924 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/573 2024-04-01T03:42:34.924 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/573 2024-04-01T03:42:34.924 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:34.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:34.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:34.934 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2024-04-01T03:42:35.110 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:35.114 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:35.114 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.230 INFO:teuthology.orchestra.run.smithi064.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:35.234 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/574 2024-04-01T03:42:35.234 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/574 2024-04-01T03:42:35.234 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:35.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:35.244 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2024-04-01T03:42:35.431 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:35.435 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:35.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.550 INFO:teuthology.orchestra.run.smithi064.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/575 2024-04-01T03:42:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/575 2024-04-01T03:42:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:35.556 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:35.564 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2024-04-01T03:42:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:35.743 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.895 INFO:teuthology.orchestra.run.smithi064.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:35.899 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/576 2024-04-01T03:42:35.899 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/576 2024-04-01T03:42:35.899 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:35.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:35.909 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2024-04-01T03:42:36.082 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:36.086 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.208 INFO:teuthology.orchestra.run.smithi064.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:36.212 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/577 2024-04-01T03:42:36.212 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/577 2024-04-01T03:42:36.212 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.222 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:36.222 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2024-04-01T03:42:36.396 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:36.400 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:36.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.551 INFO:teuthology.orchestra.run.smithi064.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-01T03:42:36.554 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/578 2024-04-01T03:42:36.554 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/578 2024-04-01T03:42:36.554 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:36.557 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:36.565 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2024-04-01T03:42:36.751 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:36.755 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:36.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.881 INFO:teuthology.orchestra.run.smithi064.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:36.884 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/579 2024-04-01T03:42:36.884 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/579 2024-04-01T03:42:36.884 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:36.886 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:36.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:36.895 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2024-04-01T03:42:37.076 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:37.080 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:37.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:37.252 INFO:teuthology.orchestra.run.smithi064.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:37.255 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/580 2024-04-01T03:42:37.255 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/580 2024-04-01T03:42:37.255 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:37.257 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:37.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:37.266 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2024-04-01T03:42:37.450 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:37.454 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:37.454 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.075 INFO:teuthology.orchestra.run.smithi064.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:38.079 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/581 2024-04-01T03:42:38.079 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/581 2024-04-01T03:42:38.079 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:38.090 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2024-04-01T03:42:38.278 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:38.282 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:38.282 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.426 INFO:teuthology.orchestra.run.smithi064.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:38.429 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/582 2024-04-01T03:42:38.429 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/582 2024-04-01T03:42:38.429 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:38.440 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2024-04-01T03:42:38.613 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:38.617 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.761 INFO:teuthology.orchestra.run.smithi064.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:38.765 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/583 2024-04-01T03:42:38.765 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/583 2024-04-01T03:42:38.765 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:38.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:38.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:38.776 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2024-04-01T03:42:38.950 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:38.954 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:38.954 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:39.096 INFO:teuthology.orchestra.run.smithi064.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:39.100 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/584 2024-04-01T03:42:39.100 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/584 2024-04-01T03:42:39.100 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:39.102 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:39.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:39.111 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2024-04-01T03:42:39.293 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:39.297 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:39.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:39.423 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/585 2024-04-01T03:42:39.423 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/585 2024-04-01T03:42:39.423 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:39.425 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:39.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:39.433 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2024-04-01T03:42:39.611 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:39.615 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:39.615 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:56.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:42:56.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:42:58.205 INFO:teuthology.orchestra.run.smithi064.stdout:generic/586 19s 2024-04-01T03:42:58.210 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/586 2024-04-01T03:42:58.210 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:58.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:58.221 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2024-04-01T03:42:58.686 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:58.690 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:58.948 INFO:teuthology.orchestra.run.smithi064.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T03:42:58.952 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/587 2024-04-01T03:42:58.952 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/587 2024-04-01T03:42:58.952 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:58.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:58.963 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2024-04-01T03:42:59.142 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:59.146 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.274 INFO:teuthology.orchestra.run.smithi064.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:59.277 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/588 2024-04-01T03:42:59.278 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/588 2024-04-01T03:42:59.278 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:59.289 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2024-04-01T03:42:59.468 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:59.472 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:59.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.612 INFO:teuthology.orchestra.run.smithi064.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:59.615 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/589 2024-04-01T03:42:59.615 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/589 2024-04-01T03:42:59.615 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:59.626 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2024-04-01T03:42:59.811 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:42:59.815 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:42:59.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.928 INFO:teuthology.orchestra.run.smithi064.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:42:59.931 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/590 2024-04-01T03:42:59.931 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/590 2024-04-01T03:42:59.932 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:42:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:42:59.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:42:59.942 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2024-04-01T03:43:00.117 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:00.121 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:00.391 INFO:teuthology.orchestra.run.smithi064.stdout:generic/591 0s 2024-04-01T03:43:00.396 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/591 2024-04-01T03:43:00.396 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:00.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:00.407 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2024-04-01T03:43:00.856 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:00.861 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:00.861 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.014 INFO:teuthology.orchestra.run.smithi064.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:01.018 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/592 2024-04-01T03:43:01.018 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/592 2024-04-01T03:43:01.018 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:01.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:01.029 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2024-04-01T03:43:01.198 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:01.202 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:01.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.347 INFO:teuthology.orchestra.run.smithi064.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/593 2024-04-01T03:43:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/593 2024-04-01T03:43:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:01.352 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:01.361 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2024-04-01T03:43:01.536 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:01.539 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:01.540 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.671 INFO:teuthology.orchestra.run.smithi064.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:01.674 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/594 2024-04-01T03:43:01.674 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/594 2024-04-01T03:43:01.674 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:01.676 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:01.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:01.685 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2024-04-01T03:43:01.864 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:01.868 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:01.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/595 2024-04-01T03:43:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/595 2024-04-01T03:43:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:02.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:02.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:02.028 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2024-04-01T03:43:02.198 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:02.202 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:02.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:02.379 INFO:teuthology.orchestra.run.smithi064.stdout:generic/596 [not run] file system doesn't support chattr +S 2024-04-01T03:43:02.383 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/596 2024-04-01T03:43:02.383 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/596 2024-04-01T03:43:02.383 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:02.385 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:02.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:02.393 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2024-04-01T03:43:02.577 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:02.581 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:02.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:03.220 INFO:teuthology.orchestra.run.smithi064.stdout:generic/597 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/597.out.bad) 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/597.out 2024-04-01 00:14:57.034907703 +0000 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/597.out.bad 2024-04-01 03:43:03.199535049 +0000 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -4,11 +4,11 @@ 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_symlinks = 0 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: successfully followed symlink 2024-04-01T03:43:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_symlinks = 1 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: +successfully followed symlink 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: == Test hardlink create protection when 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: == process != target owner and process cannot read target 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_hardlinks = 0 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: -successfully created hardlink 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: +Permission denied 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_hardlinks = 1 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2024-04-01T03:43:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: +Permission denied 2024-04-01T03:43:03.224 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/597 2024-04-01T03:43:03.224 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/597 2024-04-01T03:43:03.224 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T03:43:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:03.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T03:43:03.257 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T03:43:03.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T03:43:03.257 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2024-04-01T03:43:03.690 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:03.694 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:04.349 INFO:teuthology.orchestra.run.smithi064.stdout:generic/598 - output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/598.out.bad) 2024-04-01T03:43:04.350 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/598.out 2024-04-01 00:14:57.034907703 +0000 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/598.out.bad 2024-04-01 03:43:04.347512489 +0000 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -5,13 +5,10 @@ 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_regular = 1 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: = group & world writable dir 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_regular = 2 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: = group & world writable dir 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.351 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_fifos = 0 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -19,10 +16,7 @@ 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_fifos = 1 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: = group & world writable dir 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: fs.protected_fifos = 2 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: = group & world writable dir 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.352 INFO:teuthology.orchestra.run.smithi064.stdout: = only group writable dir 2024-04-01T03:43:04.353 INFO:teuthology.orchestra.run.smithi064.stdout: -Permission denied 2024-04-01T03:43:04.353 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/598 2024-04-01T03:43:04.353 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/598 2024-04-01T03:43:04.353 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T03:43:04.355 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:04.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T03:43:04.364 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T03:43:04.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T03:43:04.365 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2024-04-01T03:43:04.817 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:04.821 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:04.821 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:04.942 INFO:teuthology.orchestra.run.smithi064.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:04.945 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/599 2024-04-01T03:43:04.945 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/599 2024-04-01T03:43:04.945 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:04.948 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:04.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:04.956 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2024-04-01T03:43:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:05.145 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:05.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.261 INFO:teuthology.orchestra.run.smithi064.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:05.264 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/600 2024-04-01T03:43:05.265 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/600 2024-04-01T03:43:05.265 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:05.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:05.275 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2024-04-01T03:43:05.451 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:05.455 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:05.455 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.575 INFO:teuthology.orchestra.run.smithi064.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:05.578 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/601 2024-04-01T03:43:05.578 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/601 2024-04-01T03:43:05.578 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:05.580 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:05.589 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2024-04-01T03:43:05.766 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:05.770 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:05.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.916 INFO:teuthology.orchestra.run.smithi064.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:05.919 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/602 2024-04-01T03:43:05.919 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/602 2024-04-01T03:43:05.919 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:05.921 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:05.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:05.930 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2024-04-01T03:43:06.102 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:06.106 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:06.107 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.232 INFO:teuthology.orchestra.run.smithi064.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:06.235 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/603 2024-04-01T03:43:06.235 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/603 2024-04-01T03:43:06.235 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:06.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:06.246 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2024-04-01T03:43:06.422 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:06.426 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:06.426 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:06.552 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/604 2024-04-01T03:43:06.552 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/604 2024-04-01T03:43:06.552 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:06.554 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:06.563 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2024-04-01T03:43:06.750 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:06.754 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:06.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.884 INFO:teuthology.orchestra.run.smithi064.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:06.888 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/605 2024-04-01T03:43:06.888 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/605 2024-04-01T03:43:06.888 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:06.890 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:06.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:06.910 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2024-04-01T03:43:07.085 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:07.089 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:07.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.225 INFO:teuthology.orchestra.run.smithi064.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:07.228 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/606 2024-04-01T03:43:07.228 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/606 2024-04-01T03:43:07.228 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:07.230 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:07.238 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2024-04-01T03:43:07.408 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:07.412 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:07.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.528 INFO:teuthology.orchestra.run.smithi064.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:07.532 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/607 2024-04-01T03:43:07.532 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/607 2024-04-01T03:43:07.532 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:07.534 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:07.543 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2024-04-01T03:43:07.725 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:07.729 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:07.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.843 INFO:teuthology.orchestra.run.smithi064.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:07.846 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/608 2024-04-01T03:43:07.847 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/608 2024-04-01T03:43:07.847 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:07.849 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:07.856 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:07.857 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2024-04-01T03:43:08.030 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:08.034 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:08.034 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:08.403 INFO:teuthology.orchestra.run.smithi064.stdout:generic/609 0s 2024-04-01T03:43:08.409 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/609 2024-04-01T03:43:08.409 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:08.411 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:08.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:08.419 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2024-04-01T03:43:08.859 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:08.863 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:08.863 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:08.983 INFO:teuthology.orchestra.run.smithi064.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:08.987 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/610 2024-04-01T03:43:08.987 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/610 2024-04-01T03:43:08.987 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:08.989 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:08.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:08.998 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2024-04-01T03:43:09.171 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:09.175 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:09.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:09.291 INFO:teuthology.orchestra.run.smithi064.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:09.295 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/611 2024-04-01T03:43:09.295 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/611 2024-04-01T03:43:09.295 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:09.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:09.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:09.305 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2024-04-01T03:43:09.480 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:09.484 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:09.648 INFO:teuthology.orchestra.run.smithi064.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T03:43:09.652 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/612 2024-04-01T03:43:09.652 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/612 2024-04-01T03:43:09.652 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:09.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:09.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:09.663 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2024-04-01T03:43:10.132 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:10.136 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:10.136 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.325 INFO:teuthology.orchestra.run.smithi064.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:10.329 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/613 2024-04-01T03:43:10.329 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/613 2024-04-01T03:43:10.329 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:10.331 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:10.339 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2024-04-01T03:43:10.521 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:10.525 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:10.525 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.640 INFO:teuthology.orchestra.run.smithi064.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:10.643 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/614 2024-04-01T03:43:10.643 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/614 2024-04-01T03:43:10.643 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:10.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:10.654 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2024-04-01T03:43:10.829 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:10.832 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:10.833 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.953 INFO:teuthology.orchestra.run.smithi064.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:10.956 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/615 2024-04-01T03:43:10.957 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/615 2024-04-01T03:43:10.957 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:10.959 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:10.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:10.976 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2024-04-01T03:43:11.153 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:11.157 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:11.157 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.281 INFO:teuthology.orchestra.run.smithi064.stdout:generic/616 [not run] kernel does not support IO_URING 2024-04-01T03:43:11.284 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/616 2024-04-01T03:43:11.284 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/616 2024-04-01T03:43:11.284 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:11.287 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:11.295 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2024-04-01T03:43:11.469 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:11.473 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:11.473 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.638 INFO:teuthology.orchestra.run.smithi064.stdout:generic/617 [not run] kernel does not support IO_URING 2024-04-01T03:43:11.641 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/617 2024-04-01T03:43:11.641 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/617 2024-04-01T03:43:11.641 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:11.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:11.652 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2024-04-01T03:43:11.834 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:11.838 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:11.838 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.954 INFO:teuthology.orchestra.run.smithi064.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:11.958 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/618 2024-04-01T03:43:11.958 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/618 2024-04-01T03:43:11.958 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:11.960 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:11.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:11.985 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2024-04-01T03:43:12.160 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:12.164 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.290 INFO:teuthology.orchestra.run.smithi064.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:12.293 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/619 2024-04-01T03:43:12.294 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/619 2024-04-01T03:43:12.294 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:12.296 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:12.304 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2024-04-01T03:43:12.477 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:12.481 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.633 INFO:teuthology.orchestra.run.smithi064.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:12.636 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/620 2024-04-01T03:43:12.636 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/620 2024-04-01T03:43:12.637 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:12.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.646 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:12.647 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2024-04-01T03:43:12.825 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:12.829 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:12.829 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.967 INFO:teuthology.orchestra.run.smithi064.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:12.970 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/621 2024-04-01T03:43:12.971 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/621 2024-04-01T03:43:12.971 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:12.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:12.999 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2024-04-01T03:43:13.173 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:13.177 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:13.290 INFO:teuthology.orchestra.run.smithi064.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:13.294 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/622 2024-04-01T03:43:13.294 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/622 2024-04-01T03:43:13.294 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:13.296 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:13.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:13.304 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2024-04-01T03:43:13.479 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:13.483 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:13.483 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:13.694 INFO:teuthology.orchestra.run.smithi064.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:13.698 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/623 2024-04-01T03:43:13.698 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/623 2024-04-01T03:43:13.698 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:13.700 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:13.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:13.709 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2024-04-01T03:43:13.889 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:13.893 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:13.893 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.010 INFO:teuthology.orchestra.run.smithi064.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:14.013 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/624 2024-04-01T03:43:14.013 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/624 2024-04-01T03:43:14.014 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:14.016 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:14.035 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2024-04-01T03:43:14.212 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:14.216 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:14.216 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.361 INFO:teuthology.orchestra.run.smithi064.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:14.365 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/625 2024-04-01T03:43:14.365 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/625 2024-04-01T03:43:14.365 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:14.376 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2024-04-01T03:43:14.551 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:14.554 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:14.555 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.690 INFO:teuthology.orchestra.run.smithi064.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:14.693 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/626 2024-04-01T03:43:14.693 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/626 2024-04-01T03:43:14.693 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:14.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:14.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:14.706 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2024-04-01T03:43:14.882 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:14.886 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:14.886 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.007 INFO:teuthology.orchestra.run.smithi064.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:15.010 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/627 2024-04-01T03:43:15.011 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/627 2024-04-01T03:43:15.011 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:15.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:15.038 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2024-04-01T03:43:15.213 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:15.216 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.348 INFO:teuthology.orchestra.run.smithi064.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:15.351 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/628 2024-04-01T03:43:15.352 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/628 2024-04-01T03:43:15.352 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:15.354 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:15.362 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2024-04-01T03:43:15.536 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:15.540 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:15.540 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.664 INFO:teuthology.orchestra.run.smithi064.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:15.668 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/629 2024-04-01T03:43:15.668 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/629 2024-04-01T03:43:15.668 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:15.670 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:15.678 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2024-04-01T03:43:15.851 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:15.855 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:15.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.974 INFO:teuthology.orchestra.run.smithi064.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:15.978 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/630 2024-04-01T03:43:15.978 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/630 2024-04-01T03:43:15.978 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:15.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:15.989 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2024-04-01T03:43:16.171 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:16.175 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:16.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:16.297 INFO:teuthology.orchestra.run.smithi064.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:16.301 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/631 2024-04-01T03:43:16.301 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/631 2024-04-01T03:43:16.301 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:16.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:16.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:16.312 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2024-04-01T03:43:16.485 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:16.489 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:16.489 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:19.190 INFO:teuthology.orchestra.run.smithi064.stdout:generic/632 3s 2024-04-01T03:43:19.240 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/632 2024-04-01T03:43:19.240 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:19.242 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:19.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:19.251 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2024-04-01T03:43:19.425 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:19.429 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:19.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:19.724 INFO:teuthology.orchestra.run.smithi064.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/633.out.bad) 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: --- tests/generic/633.out 2024-04-01 00:14:57.036907663 +0000 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: +++ /tmp/tmp.ALJJcUYdNQxfstests-dev/results//generic/633.out.bad 2024-04-01 03:43:19.723210540 +0000 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: @@ -1,2 +1,5 @@ 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: QA output created by 633 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: Silence is golden 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2024-04-01T03:43:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2024-04-01T03:43:19.749 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/633 2024-04-01T03:43:19.749 INFO:teuthology.orchestra.run.smithi064.stdout:Failures: generic/633 2024-04-01T03:43:19.749 INFO:teuthology.orchestra.run.smithi064.stdout:Failed 1 of 1 tests 2024-04-01T03:43:19.753 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:19.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T03:43:19.766 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-01T03:43:19.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-01T03:43:19.766 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2024-04-01T03:43:20.174 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:20.179 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.296 INFO:teuthology.orchestra.run.smithi064.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:20.300 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/634 2024-04-01T03:43:20.300 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/634 2024-04-01T03:43:20.300 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:20.311 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2024-04-01T03:43:20.496 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:20.500 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:20.500 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.620 INFO:teuthology.orchestra.run.smithi064.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:20.623 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/635 2024-04-01T03:43:20.623 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/635 2024-04-01T03:43:20.623 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:20.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:20.634 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2024-04-01T03:43:20.805 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:20.809 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:20.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.925 INFO:teuthology.orchestra.run.smithi064.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:20.928 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/636 2024-04-01T03:43:20.928 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/636 2024-04-01T03:43:20.928 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:20.931 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:20.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:20.939 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2024-04-01T03:43:21.115 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:21.119 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:21.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:22.243 INFO:teuthology.orchestra.run.smithi064.stdout:generic/637 1s 2024-04-01T03:43:22.248 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/637 2024-04-01T03:43:22.248 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:22.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:22.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:22.259 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2024-04-01T03:43:22.745 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:22.750 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:22.750 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:22.916 INFO:teuthology.orchestra.run.smithi064.stdout:generic/638 0s 2024-04-01T03:43:22.922 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/638 2024-04-01T03:43:22.922 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:22.924 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:22.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:22.932 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2024-04-01T03:43:23.108 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:23.112 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:23.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.030 INFO:teuthology.orchestra.run.smithi064.stdout:generic/639 2s 2024-04-01T03:43:25.036 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/639 2024-04-01T03:43:25.036 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:25.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:25.048 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2024-04-01T03:43:25.236 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:25.240 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:25.240 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.368 INFO:teuthology.orchestra.run.smithi064.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:25.372 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/640 2024-04-01T03:43:25.372 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/640 2024-04-01T03:43:25.372 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:25.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:25.383 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2024-04-01T03:43:25.564 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:25.568 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:25.568 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.688 INFO:teuthology.orchestra.run.smithi064.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:25.691 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/641 2024-04-01T03:43:25.691 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/641 2024-04-01T03:43:25.691 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:25.702 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:25.702 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2024-04-01T03:43:25.877 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:25.881 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:25.881 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.002 INFO:teuthology.orchestra.run.smithi064.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:26.006 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/642 2024-04-01T03:43:26.006 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/642 2024-04-01T03:43:26.006 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:26.008 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:26.017 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2024-04-01T03:43:26.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:43:26.140 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:43:26.198 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:26.202 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:26.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.329 INFO:teuthology.orchestra.run.smithi064.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:26.332 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/643 2024-04-01T03:43:26.332 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/643 2024-04-01T03:43:26.332 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:26.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2024-04-01T03:43:26.571 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:26.575 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:26.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2024-04-01T03:43:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/644 2024-04-01T03:43:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/644 2024-04-01T03:43:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:26.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:26.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:26.713 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2024-04-01T03:43:26.892 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:26.896 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:26.896 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.014 INFO:teuthology.orchestra.run.smithi064.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2024-04-01T03:43:27.017 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/645 2024-04-01T03:43:27.018 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/645 2024-04-01T03:43:27.018 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:27.029 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2024-04-01T03:43:27.207 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:27.211 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:27.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/646 2024-04-01T03:43:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/646 2024-04-01T03:43:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:27.331 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:27.339 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2024-04-01T03:43:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.787 INFO:teuthology.orchestra.run.smithi064.stdout:generic/647 0s 2024-04-01T03:43:27.792 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/647 2024-04-01T03:43:27.792 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:27.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:27.803 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2024-04-01T03:43:28.261 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:28.265 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:28.398 INFO:teuthology.orchestra.run.smithi064.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T03:43:28.401 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/648 2024-04-01T03:43:28.401 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/648 2024-04-01T03:43:28.401 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:28.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:28.413 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2024-04-01T03:43:28.595 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:28.599 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:28.765 INFO:teuthology.orchestra.run.smithi064.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T03:43:28.768 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/649 2024-04-01T03:43:28.768 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/649 2024-04-01T03:43:28.768 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T03:43:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:28.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T03:43:28.779 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2024-04-01T03:43:29.233 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T03:43:29.238 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T03:43:29.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T03:43:56.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:43:56.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:44:26.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:44:26.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:44:56.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:44:56.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:45:26.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:45:26.607 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:45:56.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:45:56.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:46:26.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:46:26.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:46:57.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:46:57.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:47:27.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:47:27.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:47:57.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:47:57.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:48:27.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:48:27.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:48:58.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:48:58.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:49:28.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:49:28.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:49:58.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:49:58.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:50:28.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:50:28.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:50:58.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:50:58.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:51:28.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:51:28.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:51:58.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:51:58.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:51:59.012 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T03:51:59.011+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 51417) UID: 0 2024-04-01T03:51:59.014 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T03:51:59.014+0000 7f780466b640 -1 received signal: Hangup from (PID: 51417) UID: 0 2024-04-01T03:51:59.104 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T03:51:59.103+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 512900) UID: 0 2024-04-01T03:51:59.106 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T03:51:59.104+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 512900) UID: 0 2024-04-01T03:51:59.109 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T03:51:59.108+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 512900) UID: 0 2024-04-01T03:51:59.127 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T03:51:59.126+0000 7f4548999640 -1 Fail to open '/proc/512900/cmdline' error = (2) No such file or directory 2024-04-01T03:51:59.135 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T03:51:59.126+0000 7f4548999640 -1 received signal: Hangup from (PID: 512900) UID: 0 2024-04-01T03:56:26.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:56:26.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:56:56.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:56:56.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:57:26.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:57:26.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:57:56.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:57:56.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:58:27.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:58:27.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:58:57.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:58:57.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:59:27.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:59:27.235 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:59:57.351 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T03:59:57.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:00:27.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:00:27.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:00:57.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:00:57.688 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:01:27.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:01:27.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:01:57.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:01:57.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:02:28.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:02:28.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:02:58.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:02:58.243 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:03:28.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:03:28.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:03:58.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:03:58.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:04:28.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:04:28.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:04:58.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:04:58.741 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:05:28.903 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:05:28.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:05:59.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:05:59.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:06:29.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:06:29.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:06:59.365 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:06:59.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:06:59.410 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T04:06:59.409+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51860) UID: 0 2024-04-01T04:06:59.410 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T04:06:59.409+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51860) UID: 0 2024-04-01T04:06:59.410 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T04:06:59.409+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51860) UID: 0 2024-04-01T04:06:59.411 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T04:06:59.409+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51860) UID: 0 2024-04-01T04:09:49.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:09:49.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:10:19.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:10:19.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:10:49.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:10:49.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:11:19.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:11:19.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:11:49.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:11:49.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:12:19.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:12:19.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:12:50.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:12:50.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:13:20.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:13:20.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:13:50.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:13:50.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:14:20.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:14:20.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:14:50.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:14:50.925 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:15:21.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:15:21.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:15:51.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:15:51.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:16:21.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:16:21.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:16:21.476 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T04:16:21.475+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 52133) UID: 0 2024-04-01T04:16:21.476 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T04:16:21.475+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 52133) UID: 0 2024-04-01T04:19:20.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:19:20.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:19:51.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:19:51.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:19:51.385 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T04:19:51.383+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52182) UID: 0 2024-04-01T04:19:51.385 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T04:19:51.383+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52182) UID: 0 2024-04-01T04:19:51.385 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T04:19:51.383+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52182) UID: 0 2024-04-01T04:19:51.386 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T04:19:51.383+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52182) UID: 0 2024-04-01T04:22:42.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:22:42.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:23:12.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:23:12.274 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:23:42.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:23:42.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:24:12.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:24:12.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:24:42.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:24:42.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:25:13.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:25:13.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:25:43.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:25:43.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:26:13.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:26:13.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:26:43.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:26:43.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:27:13.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:27:13.913 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:27:44.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:27:44.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:28:14.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:28:14.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:28:44.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:28:44.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:29:14.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:29:14.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:29:44.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:29:44.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:30:14.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:30:14.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:30:45.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:30:45.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:31:15.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:31:15.504 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:31:45.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:31:45.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:32:15.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:32:15.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:32:45.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:32:45.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:33:16.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:33:16.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:33:46.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:33:46.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:34:16.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:34:16.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:34:46.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:34:46.560 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:35:17.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:35:17.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:35:47.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:35:47.145 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:36:17.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:36:17.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:36:47.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:36:47.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:37:17.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:37:17.939 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:37:48.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:37:48.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:38:18.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:38:18.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:38:18.283 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T04:38:18.282+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 52799) UID: 0 2024-04-01T04:38:18.284 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T04:38:18.283+0000 7f2581580640 -1 Fail to open '/proc/52799/cmdline' error = (2) No such file or directory 2024-04-01T04:38:18.284 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T04:38:18.283+0000 7f2581580640 -1 received signal: Hangup from (PID: 52799) UID: 0 2024-04-01T04:41:18.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:41:18.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:41:48.929 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:41:48.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:42:19.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:42:19.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:42:49.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:42:49.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:43:19.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:43:19.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:43:50.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:43:50.026 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:44:20.231 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:44:20.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:44:20.333 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T04:44:20.331+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 522792) UID: 0 2024-04-01T04:44:20.334 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T04:44:20.333+0000 7fd22327d640 -1 Fail to open '/proc/522792/cmdline' error = (2) No such file or directory 2024-04-01T04:44:20.334 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T04:44:20.333+0000 7fd22327d640 -1 received signal: Hangup from (PID: 522792) UID: 0 2024-04-01T04:44:20.340 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T04:44:20.331+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 522792) UID: 0 2024-04-01T04:44:20.354 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T04:44:20.331+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 522792) UID: 0 2024-04-01T04:48:13.064 INFO:teuthology.orchestra.run.smithi064.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.3.log.1: 2024-04-01T04:48:13.064 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: stdin: file size changed while zipping 2024-04-01T04:48:45.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:48:45.011 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:49:15.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:49:15.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:49:45.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:49:45.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:50:15.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:50:15.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:50:45.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:50:45.598 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:51:15.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:51:15.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:51:45.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:51:45.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:51:45.934 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T04:51:45.932+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53078) UID: 0 2024-04-01T04:51:45.935 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T04:51:45.932+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53078) UID: 0 2024-04-01T04:51:45.935 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T04:51:45.932+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53078) UID: 0 2024-04-01T04:51:45.935 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T04:51:45.932+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53078) UID: 0 2024-04-01T04:55:05.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:55:05.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:55:35.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:55:35.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:56:05.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:56:05.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:56:05.311 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T04:56:05.310+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53140) UID: 0 2024-04-01T04:56:05.311 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T04:56:05.310+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53140) UID: 0 2024-04-01T04:56:25.989 INFO:teuthology.orchestra.run.smithi064.stdout:generic/650 4376s 2024-04-01T04:56:26.011 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/650 2024-04-01T04:56:26.011 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:26.021 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:26.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:26.047 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/651) 2024-04-01T04:56:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:26.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:27.004 INFO:teuthology.orchestra.run.smithi064.stdout:generic/651 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:27.009 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/651 2024-04-01T04:56:27.009 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/651 2024-04-01T04:56:27.009 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:27.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:27.049 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/652) 2024-04-01T04:56:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:27.602 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:generic/652 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:27.957 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/652 2024-04-01T04:56:27.957 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/652 2024-04-01T04:56:27.957 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:27.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:27.974 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/653) 2024-04-01T04:56:28.452 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:28.465 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:28.762 INFO:teuthology.orchestra.run.smithi064.stdout:generic/653 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:28.771 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/653 2024-04-01T04:56:28.771 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/653 2024-04-01T04:56:28.771 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:28.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:28.808 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/654) 2024-04-01T04:56:29.354 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:29.362 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:29.363 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:29.726 INFO:teuthology.orchestra.run.smithi064.stdout:generic/654 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:29.733 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/654 2024-04-01T04:56:29.733 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/654 2024-04-01T04:56:29.734 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:29.738 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:29.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:29.751 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/655) 2024-04-01T04:56:30.273 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:30.284 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:30.284 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:30.654 INFO:teuthology.orchestra.run.smithi064.stdout:generic/655 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:30.659 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/655 2024-04-01T04:56:30.659 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/655 2024-04-01T04:56:30.659 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:30.666 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:30.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:30.689 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/656) 2024-04-01T04:56:31.215 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:31.220 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:31.220 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:31.595 INFO:teuthology.orchestra.run.smithi064.stdout:generic/656 [not run] vfstest not support by ceph-fuse 2024-04-01T04:56:31.609 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/656 2024-04-01T04:56:31.609 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/656 2024-04-01T04:56:31.609 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:31.615 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:31.638 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:31.638 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/657) 2024-04-01T04:56:32.100 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:32.105 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:32.106 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:32.414 INFO:teuthology.orchestra.run.smithi064.stdout:generic/657 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:32.421 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/657 2024-04-01T04:56:32.421 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/657 2024-04-01T04:56:32.422 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:32.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:32.452 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/658) 2024-04-01T04:56:32.968 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:32.981 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:33.282 INFO:teuthology.orchestra.run.smithi064.stdout:generic/658 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:33.293 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/658 2024-04-01T04:56:33.294 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/658 2024-04-01T04:56:33.294 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:33.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:33.323 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/659) 2024-04-01T04:56:33.899 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:33.911 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:33.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:34.265 INFO:teuthology.orchestra.run.smithi064.stdout:generic/659 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:34.277 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/659 2024-04-01T04:56:34.278 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/659 2024-04-01T04:56:34.278 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:34.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:34.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:34.312 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/660) 2024-04-01T04:56:34.831 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:34.846 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:35.165 INFO:teuthology.orchestra.run.smithi064.stdout:generic/660 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:35.174 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/660 2024-04-01T04:56:35.174 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/660 2024-04-01T04:56:35.175 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:35.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:35.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:35.201 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/661) 2024-04-01T04:56:35.850 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:35.858 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:35.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:36.175 INFO:teuthology.orchestra.run.smithi064.stdout:generic/661 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:36.183 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/661 2024-04-01T04:56:36.184 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/661 2024-04-01T04:56:36.184 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:36.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:36.213 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/662) 2024-04-01T04:56:36.740 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:36.755 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:36.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:37.228 INFO:teuthology.orchestra.run.smithi064.stdout:generic/662 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:37.231 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/662 2024-04-01T04:56:37.231 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/662 2024-04-01T04:56:37.232 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:37.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:37.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:37.248 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/663) 2024-04-01T04:56:37.818 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:37.840 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:37.841 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:38.217 INFO:teuthology.orchestra.run.smithi064.stdout:generic/663 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:38.227 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/663 2024-04-01T04:56:38.227 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/663 2024-04-01T04:56:38.227 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:38.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:38.267 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/664) 2024-04-01T04:56:38.888 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:38.915 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:38.915 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:39.282 INFO:teuthology.orchestra.run.smithi064.stdout:generic/664 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:39.288 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/664 2024-04-01T04:56:39.289 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/664 2024-04-01T04:56:39.289 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:39.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:39.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:39.323 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/665) 2024-04-01T04:56:39.961 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:39.980 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:39.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:40.331 INFO:teuthology.orchestra.run.smithi064.stdout:generic/665 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:40.343 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/665 2024-04-01T04:56:40.344 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/665 2024-04-01T04:56:40.344 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:40.354 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:40.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:40.376 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/666) 2024-04-01T04:56:40.992 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:41.005 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:41.005 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:41.341 INFO:teuthology.orchestra.run.smithi064.stdout:generic/666 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:41.353 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/666 2024-04-01T04:56:41.353 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/666 2024-04-01T04:56:41.353 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:41.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:41.366 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/667) 2024-04-01T04:56:41.868 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:41.879 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:42.227 INFO:teuthology.orchestra.run.smithi064.stdout:generic/667 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:42.230 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/667 2024-04-01T04:56:42.231 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/667 2024-04-01T04:56:42.231 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:42.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:42.256 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/668) 2024-04-01T04:56:42.831 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:42.846 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:43.221 INFO:teuthology.orchestra.run.smithi064.stdout:generic/668 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:43.230 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/668 2024-04-01T04:56:43.230 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/668 2024-04-01T04:56:43.230 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:43.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:43.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:43.266 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/669) 2024-04-01T04:56:43.836 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:43.842 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:44.269 INFO:teuthology.orchestra.run.smithi064.stdout:generic/669 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/669 2024-04-01T04:56:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/669 2024-04-01T04:56:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:44.287 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:44.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:44.310 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/670) 2024-04-01T04:56:44.819 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:44.837 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:44.837 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:45.182 INFO:teuthology.orchestra.run.smithi064.stdout:generic/670 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:45.194 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/670 2024-04-01T04:56:45.194 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/670 2024-04-01T04:56:45.195 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:45.197 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:45.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:45.211 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/671) 2024-04-01T04:56:45.687 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:45.694 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:45.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:46.032 INFO:teuthology.orchestra.run.smithi064.stdout:generic/671 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:46.039 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/671 2024-04-01T04:56:46.040 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/671 2024-04-01T04:56:46.040 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:46.046 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:46.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:46.068 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/672) 2024-04-01T04:56:46.586 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:46.593 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:46.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:46.918 INFO:teuthology.orchestra.run.smithi064.stdout:generic/672 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:46.931 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/672 2024-04-01T04:56:46.932 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/672 2024-04-01T04:56:46.932 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:46.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:46.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:46.968 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/673) 2024-04-01T04:56:47.476 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:47.485 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:47.485 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:48.290 INFO:teuthology.orchestra.run.smithi064.stdout:generic/673 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:48.298 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/673 2024-04-01T04:56:48.299 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/673 2024-04-01T04:56:48.299 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:48.302 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:48.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:48.326 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/674) 2024-04-01T04:56:48.817 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:48.829 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:48.829 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:49.270 INFO:teuthology.orchestra.run.smithi064.stdout:generic/674 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:49.283 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/674 2024-04-01T04:56:49.283 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/674 2024-04-01T04:56:49.285 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:49.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:49.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:49.311 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/675) 2024-04-01T04:56:49.803 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:49.814 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:49.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:50.187 INFO:teuthology.orchestra.run.smithi064.stdout:generic/675 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T04:56:50.197 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/675 2024-04-01T04:56:50.197 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/675 2024-04-01T04:56:50.197 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T04:56:50.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:56:50.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T04:56:50.228 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/676) 2024-04-01T04:56:50.747 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T04:56:50.758 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T04:56:50.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T04:59:09.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:59:09.496 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:59:39.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T04:59:39.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:00:09.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:00:09.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:00:40.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:00:40.203 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:01:10.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:01:10.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:01:40.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:01:40.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:02:10.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:02:10.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:02:41.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:02:41.321 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:03:11.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:03:11.713 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:03:42.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:03:42.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:04:12.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:04:12.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:04:42.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:04:42.395 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:05:12.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:05:12.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:05:42.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:05:42.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:06:13.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:06:13.243 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:06:43.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:06:43.386 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:07:13.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:07:13.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:07:13.545 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T05:07:13.545+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53469) UID: 0 2024-04-01T05:07:13.546 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T05:07:13.545+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53469) UID: 0 2024-04-01T05:10:03.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:10:03.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:10:03.153 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T05:10:03.153+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535798) UID: 0 2024-04-01T05:10:03.154 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T05:10:03.153+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535798) UID: 0 2024-04-01T05:10:03.154 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T05:10:03.154+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535798) UID: 0 2024-04-01T05:10:03.155 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T05:10:03.154+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535798) UID: 0 2024-04-01T05:18:09.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:18:09.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:18:09.946 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T05:18:09.946+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535823) UID: 0 2024-04-01T05:18:09.947 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T05:18:09.946+0000 7f11fc45b640 -1 Fail to open '/proc/535823/cmdline' error = (2) No such file or directory 2024-04-01T05:18:09.947 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T05:18:09.946+0000 7f11fc45b640 -1 received signal: Hangup from (PID: 535823) UID: 0 2024-04-01T05:18:09.947 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T05:18:09.946+0000 7f4548999640 -1 received signal: Hangup from (PID: 535823) UID: 0 2024-04-01T05:18:09.951 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T05:18:09.946+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 535823) UID: 0 2024-04-01T05:25:57.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:25:57.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:25:57.404 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T05:25:57.405+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53545) UID: 0 2024-04-01T05:25:57.405 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T05:25:57.405+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53545) UID: 0 2024-04-01T05:31:10.235 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T05:31:10.233+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53552) UID: 0 2024-04-01T05:31:10.235 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T05:31:10.233+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53552) UID: 0 2024-04-01T05:31:10.236 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T05:31:10.233+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53552) UID: 0 2024-04-01T05:31:10.236 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T05:31:10.233+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53552) UID: 0 2024-04-01T05:34:56.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:34:56.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:34:56.184 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T05:34:56.184+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53577) UID: 0 2024-04-01T05:34:56.184 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T05:34:56.184+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53577) UID: 0 2024-04-01T05:34:56.184 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T05:34:56.184+0000 7f1aeefbd640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53577) UID: 0 2024-04-01T05:34:56.185 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T05:34:56.184+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53577) UID: 0 2024-04-01T05:38:20.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:38:20.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:38:20.095 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-01T05:38:20.095+0000 7f780466b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53603) UID: 0 2024-04-01T05:38:20.095 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-01T05:38:20.095+0000 7f2581580640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 53603) UID: 0 2024-04-01T05:41:08.399 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-01T05:41:08.399+0000 7f4e1c701640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53611) UID: 0 2024-04-01T05:41:08.400 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-01T05:41:08.399+0000 7f398fed0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53611) UID: 0 2024-04-01T05:41:08.400 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-01T05:41:08.399+0000 7f70e603b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53611) UID: 0 2024-04-01T05:41:08.401 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T05:41:08.399+0000 7f1aeefbd640 -1 Fail to open '/proc/53611/cmdline' error = (2) No such file or directory 2024-04-01T05:41:08.401 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-01T05:41:08.399+0000 7f1aeefbd640 -1 received signal: Hangup from (PID: 53611) UID: 0 2024-04-01T05:42:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:generic/676 2745s 2024-04-01T05:42:35.578 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/676 2024-04-01T05:42:35.578 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:35.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:35.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:35.611 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/677) 2024-04-01T05:42:36.537 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:36.547 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:36.547 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:36.993 INFO:teuthology.orchestra.run.smithi064.stdout:generic/677 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:42:37.005 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/677 2024-04-01T05:42:37.005 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/677 2024-04-01T05:42:37.006 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:37.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:37.035 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/678) 2024-04-01T05:42:37.598 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:37.607 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:37.608 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:38.101 INFO:teuthology.orchestra.run.smithi064.stdout:generic/678 [not run] kernel does not support IO_URING 2024-04-01T05:42:38.110 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/678 2024-04-01T05:42:38.110 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/678 2024-04-01T05:42:38.111 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:38.120 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:38.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:38.151 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/679) 2024-04-01T05:42:38.665 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:38.670 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:38.670 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:38.987 INFO:teuthology.orchestra.run.smithi064.stdout:generic/679 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:42:39.000 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/679 2024-04-01T05:42:39.000 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/679 2024-04-01T05:42:39.000 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:39.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:39.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:39.036 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/680) 2024-04-01T05:42:39.536 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:39.547 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:39.548 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:40.595 INFO:teuthology.orchestra.run.smithi064.stdout:generic/680 1s 2024-04-01T05:42:40.608 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/680 2024-04-01T05:42:40.608 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:40.616 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:40.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:40.634 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/681) 2024-04-01T05:42:41.417 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:41.428 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:41.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:41.744 INFO:teuthology.orchestra.run.smithi064.stdout:generic/681 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T05:42:41.754 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/681 2024-04-01T05:42:41.754 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/681 2024-04-01T05:42:41.754 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:41.781 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:41.782 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/682) 2024-04-01T05:42:42.299 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:42.303 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:42.679 INFO:teuthology.orchestra.run.smithi064.stdout:generic/682 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-01T05:42:42.692 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/682 2024-04-01T05:42:42.692 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/682 2024-04-01T05:42:42.692 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:42.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:42.712 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/683) 2024-04-01T05:42:43.221 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:43.234 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:43.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:43.695 INFO:teuthology.orchestra.run.smithi064.stdout:generic/683 [not run] /tmp/tmp.vihoW6DmoUtest: file alloc unit 4194304 larger than op length 65536 2024-04-01T05:42:43.702 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/683 2024-04-01T05:42:43.703 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/683 2024-04-01T05:42:43.703 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:43.706 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:43.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:43.732 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/684) 2024-04-01T05:42:44.307 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:44.318 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:44.318 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:46.870 INFO:teuthology.orchestra.run.smithi064.stdout:generic/684 [not run] /tmp/tmp.vihoW6DmoUtest: file alloc unit 4194304 larger than op length 65536 2024-04-01T05:42:46.882 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/684 2024-04-01T05:42:46.882 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/684 2024-04-01T05:42:46.883 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:46.891 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:46.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:46.916 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/685) 2024-04-01T05:42:47.494 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:47.502 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:47.502 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:50.020 INFO:teuthology.orchestra.run.smithi064.stdout:generic/685 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-01T05:42:50.027 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/685 2024-04-01T05:42:50.027 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/685 2024-04-01T05:42:50.028 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:50.032 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:50.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:50.053 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/686) 2024-04-01T05:42:50.480 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:50.495 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:50.495 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:52.921 INFO:teuthology.orchestra.run.smithi064.stdout:generic/686 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-01T05:42:52.931 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/686 2024-04-01T05:42:52.931 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/686 2024-04-01T05:42:52.932 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:52.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:52.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:52.953 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/687) 2024-04-01T05:42:53.433 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:53.448 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:53.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:55.828 INFO:teuthology.orchestra.run.smithi064.stdout:generic/687 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-01T05:42:55.837 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/687 2024-04-01T05:42:55.837 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/687 2024-04-01T05:42:55.837 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:55.845 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:55.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:55.862 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/688) 2024-04-01T05:42:56.390 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:56.398 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:56.398 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:56.893 INFO:teuthology.orchestra.run.smithi064.stdout:generic/688 [not run] /tmp/tmp.vihoW6DmoUtest: file alloc unit 4194304 larger than op length 65536 2024-04-01T05:42:56.904 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/688 2024-04-01T05:42:56.904 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/688 2024-04-01T05:42:56.904 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:56.913 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:56.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:56.936 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/689) 2024-04-01T05:42:57.427 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:57.437 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:57.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:57.821 INFO:teuthology.orchestra.run.smithi064.stdout:generic/689 [not run] vfstest not support by ceph-fuse 2024-04-01T05:42:57.830 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/689 2024-04-01T05:42:57.830 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/689 2024-04-01T05:42:57.830 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:57.837 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:57.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:57.865 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/690) 2024-04-01T05:42:58.363 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:58.376 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:58.765 INFO:teuthology.orchestra.run.smithi064.stdout:generic/690 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:42:58.773 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/690 2024-04-01T05:42:58.773 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/690 2024-04-01T05:42:58.773 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:58.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:58.803 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/691) 2024-04-01T05:42:59.265 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:42:59.278 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:42:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:59.595 INFO:teuthology.orchestra.run.smithi064.stdout:generic/691 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:42:59.599 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/691 2024-04-01T05:42:59.599 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/691 2024-04-01T05:42:59.599 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:42:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:42:59.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:42:59.622 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/692) 2024-04-01T05:43:00.162 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:00.174 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:00.531 INFO:teuthology.orchestra.run.smithi064.stdout:generic/692 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:00.540 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/692 2024-04-01T05:43:00.540 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/692 2024-04-01T05:43:00.540 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:00.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:00.571 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/693) 2024-04-01T05:43:01.083 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:01.095 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:01.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:01.502 INFO:teuthology.orchestra.run.smithi064.stdout:generic/693 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:01.514 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/693 2024-04-01T05:43:01.514 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/693 2024-04-01T05:43:01.514 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:01.518 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:01.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:01.545 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/694) 2024-04-01T05:43:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:02.069 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:03.268 INFO:teuthology.orchestra.run.smithi064.stdout:generic/694 1s 2024-04-01T05:43:03.288 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/694 2024-04-01T05:43:03.288 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:03.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:03.307 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/695) 2024-04-01T05:43:03.857 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:03.870 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:04.277 INFO:teuthology.orchestra.run.smithi064.stdout:generic/695 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:04.281 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/695 2024-04-01T05:43:04.282 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/695 2024-04-01T05:43:04.282 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:04.308 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:04.309 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/696) 2024-04-01T05:43:04.765 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:04.776 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:04.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:05.109 INFO:teuthology.orchestra.run.smithi064.stdout:generic/696 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:05.121 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/696 2024-04-01T05:43:05.121 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/696 2024-04-01T05:43:05.121 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:05.127 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:05.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:05.150 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/697) 2024-04-01T05:43:05.629 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:05.644 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:05.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:06.058 INFO:teuthology.orchestra.run.smithi064.stdout:generic/697 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-01T05:43:06.093 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/697 2024-04-01T05:43:06.093 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/697 2024-04-01T05:43:06.093 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:06.094 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:06.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:06.096 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/698) 2024-04-01T05:43:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:06.717 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:06.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:07.101 INFO:teuthology.orchestra.run.smithi064.stdout:generic/698 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:07.110 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/698 2024-04-01T05:43:07.111 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/698 2024-04-01T05:43:07.111 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:07.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:07.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:07.162 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/699) 2024-04-01T05:43:07.746 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:07.755 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:07.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:08.207 INFO:teuthology.orchestra.run.smithi064.stdout:generic/699 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:08.218 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/699 2024-04-01T05:43:08.218 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/699 2024-04-01T05:43:08.219 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:08.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:08.259 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/700) 2024-04-01T05:43:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:08.866 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:08.866 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:09.249 INFO:teuthology.orchestra.run.smithi064.stdout:generic/700 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:09.255 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/700 2024-04-01T05:43:09.255 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/700 2024-04-01T05:43:09.255 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:09.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:09.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:09.289 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/701) 2024-04-01T05:43:09.947 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:09.959 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:09.959 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:10.766 INFO:teuthology.orchestra.run.smithi064.stdout:generic/701 1s 2024-04-01T05:43:10.791 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/701 2024-04-01T05:43:10.792 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:10.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:10.846 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:10.846 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/702) 2024-04-01T05:43:11.609 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:11.623 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:generic/702 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:11.949 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/702 2024-04-01T05:43:11.949 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/702 2024-04-01T05:43:11.949 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:11.954 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:11.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:11.978 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/703) 2024-04-01T05:43:12.472 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:12.497 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:12.890 INFO:teuthology.orchestra.run.smithi064.stdout:generic/703 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:12.895 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/703 2024-04-01T05:43:12.896 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/703 2024-04-01T05:43:12.896 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:12.899 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:12.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:12.918 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/704) 2024-04-01T05:43:13.428 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:13.439 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:13.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:13.888 INFO:teuthology.orchestra.run.smithi064.stdout:generic/704 [not run] require ceph-fuse to be valid block disk 2024-04-01T05:43:13.894 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/704 2024-04-01T05:43:13.894 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/704 2024-04-01T05:43:13.894 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:13.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:13.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:13.923 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/705) 2024-04-01T05:43:14.462 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:14.473 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:14.474 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:14.821 INFO:teuthology.orchestra.run.smithi064.stdout:generic/705 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:14.828 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/705 2024-04-01T05:43:14.829 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/705 2024-04-01T05:43:14.829 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:14.832 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:14.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:14.844 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/706) 2024-04-01T05:43:15.366 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:15.379 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:15.379 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:17.756 INFO:teuthology.orchestra.run.smithi064.stdout:generic/706 2s 2024-04-01T05:43:17.771 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/706 2024-04-01T05:43:17.771 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:17.774 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:17.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:17.788 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/707) 2024-04-01T05:43:18.554 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:18.563 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:18.824 INFO:teuthology.orchestra.run.smithi064.stdout:generic/707 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:18.832 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/707 2024-04-01T05:43:18.832 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/707 2024-04-01T05:43:18.832 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:18.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:18.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:18.871 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/708) 2024-04-01T05:43:19.364 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:19.378 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:19.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:20.367 INFO:teuthology.orchestra.run.smithi064.stdout:generic/708 1s 2024-04-01T05:43:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/708 2024-04-01T05:43:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:20.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:20.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:20.413 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/709) 2024-04-01T05:43:21.195 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:21.210 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:21.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:21.675 INFO:teuthology.orchestra.run.smithi064.stdout:generic/709 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/709 2024-04-01T05:43:21.688 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/709 2024-04-01T05:43:21.688 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:21.693 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:21.745 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:21.745 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/710) 2024-04-01T05:43:22.549 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:22.564 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:22.564 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:22.964 INFO:teuthology.orchestra.run.smithi064.stdout:generic/710 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:22.971 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/710 2024-04-01T05:43:22.971 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/710 2024-04-01T05:43:22.971 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:22.982 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:23.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:23.000 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/711) 2024-04-01T05:43:23.802 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:23.816 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:23.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:24.215 INFO:teuthology.orchestra.run.smithi064.stdout:generic/711 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/711 2024-04-01T05:43:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/711 2024-04-01T05:43:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:24.229 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:24.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:24.254 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/712) 2024-04-01T05:43:25.049 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:25.061 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:25.062 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:25.479 INFO:teuthology.orchestra.run.smithi064.stdout:generic/712 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:25.487 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/712 2024-04-01T05:43:25.488 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/712 2024-04-01T05:43:25.488 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:25.496 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:25.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:25.519 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/713) 2024-04-01T05:43:26.309 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:26.317 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:generic/713 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-01T05:43:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/713 2024-04-01T05:43:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/713 2024-04-01T05:43:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:26.739 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:26.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:26.761 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/714) 2024-04-01T05:43:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:27.609 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:28.004 INFO:teuthology.orchestra.run.smithi064.stdout:generic/714 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:28.013 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/714 2024-04-01T05:43:28.014 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/714 2024-04-01T05:43:28.014 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:28.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:28.043 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/715) 2024-04-01T05:43:28.845 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:28.850 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:28.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:29.265 INFO:teuthology.orchestra.run.smithi064.stdout:generic/715 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-01T05:43:29.274 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/715 2024-04-01T05:43:29.274 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/715 2024-04-01T05:43:29.274 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:29.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:29.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:29.311 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/716) 2024-04-01T05:43:30.098 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:30.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:30.537 INFO:teuthology.orchestra.run.smithi064.stdout:generic/716 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:30.547 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/716 2024-04-01T05:43:30.547 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/716 2024-04-01T05:43:30.548 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:30.551 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:30.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:30.570 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/717) 2024-04-01T05:43:31.342 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:31.353 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:31.805 INFO:teuthology.orchestra.run.smithi064.stdout:generic/717 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:31.816 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/717 2024-04-01T05:43:31.817 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/717 2024-04-01T05:43:31.817 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:31.826 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:31.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:31.852 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/718) 2024-04-01T05:43:32.690 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:32.702 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:33.132 INFO:teuthology.orchestra.run.smithi064.stdout:generic/718 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:33.137 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/718 2024-04-01T05:43:33.137 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/718 2024-04-01T05:43:33.137 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:33.165 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:33.165 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/719) 2024-04-01T05:43:33.972 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:33.980 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:33.980 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:34.441 INFO:teuthology.orchestra.run.smithi064.stdout:generic/719 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:34.451 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/719 2024-04-01T05:43:34.451 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/719 2024-04-01T05:43:34.452 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:34.460 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:34.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:34.487 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/720) 2024-04-01T05:43:35.267 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:35.280 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:35.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:35.821 INFO:teuthology.orchestra.run.smithi064.stdout:generic/720 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:35.828 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/720 2024-04-01T05:43:35.829 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/720 2024-04-01T05:43:35.830 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:35.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:35.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:35.888 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/721) 2024-04-01T05:43:36.801 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:36.873 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:36.873 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:37.387 INFO:teuthology.orchestra.run.smithi064.stdout:generic/721 [not run] xfs_io startupdate support is missing 2024-04-01T05:43:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/721 2024-04-01T05:43:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/721 2024-04-01T05:43:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:37.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:37.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:37.433 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/722) 2024-04-01T05:43:37.985 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:37.994 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:37.994 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:38.459 INFO:teuthology.orchestra.run.smithi064.stdout:generic/722 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-01T05:43:38.468 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/722 2024-04-01T05:43:38.468 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/722 2024-04-01T05:43:38.469 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:38.476 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:38.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:38.534 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/723) 2024-04-01T05:43:39.312 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:39.324 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:39.324 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:39.791 INFO:teuthology.orchestra.run.smithi064.stdout:generic/723 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-01T05:43:39.802 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/723 2024-04-01T05:43:39.802 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/723 2024-04-01T05:43:39.802 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:39.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:39.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:39.840 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/724) 2024-04-01T05:43:40.639 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:40.650 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:40.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout:generic/724 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-01T05:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/724 2024-04-01T05:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/724 2024-04-01T05:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:41.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:41.162 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/725) 2024-04-01T05:43:41.999 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:42.008 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:42.400 INFO:teuthology.orchestra.run.smithi064.stdout:generic/725 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-01T05:43:42.412 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/725 2024-04-01T05:43:42.413 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/725 2024-04-01T05:43:42.413 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:42.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:42.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/726) 2024-04-01T05:43:43.181 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:43.187 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:43.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:44.142 INFO:teuthology.orchestra.run.smithi064.stdout:generic/726 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-01T05:43:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/726 2024-04-01T05:43:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/726 2024-04-01T05:43:44.146 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:44.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:44.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:44.166 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/727) 2024-04-01T05:43:45.038 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:45.052 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:45.052 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:45.570 INFO:teuthology.orchestra.run.smithi064.stdout:generic/727 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-01T05:43:45.580 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/727 2024-04-01T05:43:45.580 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/727 2024-04-01T05:43:45.580 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:45.588 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:45.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:45.616 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/728) 2024-04-01T05:43:46.470 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:46.480 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:51.026 INFO:teuthology.orchestra.run.smithi064.stdout:generic/728 5s 2024-04-01T05:43:51.043 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/728 2024-04-01T05:43:51.043 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:51.052 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:51.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:51.090 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/729) 2024-04-01T05:43:51.619 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:51.633 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:51.633 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:52.569 INFO:teuthology.orchestra.run.smithi064.stdout:generic/729 1s 2024-04-01T05:43:52.584 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/729 2024-04-01T05:43:52.584 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:52.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:52.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:52.614 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/730) 2024-04-01T05:43:53.368 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:53.377 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:53.377 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:53.713 INFO:teuthology.orchestra.run.smithi064.stdout:generic/730 [not run] require ceph-fuse to be valid block disk 2024-04-01T05:43:53.723 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/730 2024-04-01T05:43:53.723 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/730 2024-04-01T05:43:53.723 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:53.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:53.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:53.751 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/731) 2024-04-01T05:43:54.298 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:54.311 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:54.311 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:54.634 INFO:teuthology.orchestra.run.smithi064.stdout:generic/731 [not run] require ceph-fuse to be valid block disk 2024-04-01T05:43:54.638 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/731 2024-04-01T05:43:54.638 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/731 2024-04-01T05:43:54.638 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:54.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:54.668 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/732) 2024-04-01T05:43:55.199 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:55.213 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:55.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:55.687 INFO:teuthology.orchestra.run.smithi064.stdout:generic/732 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:55.697 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/732 2024-04-01T05:43:55.697 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/732 2024-04-01T05:43:55.697 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:55.702 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:55.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:55.727 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/733) 2024-04-01T05:43:56.222 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:56.232 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:56.232 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:56.563 INFO:teuthology.orchestra.run.smithi064.stdout:generic/733 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:43:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/733 2024-04-01T05:43:56.575 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/733 2024-04-01T05:43:56.575 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:56.583 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:56.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:56.606 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/734) 2024-04-01T05:43:57.149 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:57.157 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:57.618 INFO:teuthology.orchestra.run.smithi064.stdout:generic/734 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-01T05:43:57.628 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/734 2024-04-01T05:43:57.628 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/734 2024-04-01T05:43:57.630 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:43:57.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:43:57.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:43:57.671 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/735) 2024-04-01T05:43:58.434 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:43:58.442 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:43:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:44:00.855 INFO:teuthology.orchestra.run.smithi064.stdout:generic/735 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-01T05:44:00.863 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/735 2024-04-01T05:44:00.863 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/735 2024-04-01T05:44:00.863 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:44:00.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:44:00.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:44:00.879 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/736) 2024-04-01T05:44:01.694 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:44:01.707 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:44:01.707 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:44:27.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:44:27.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:44:57.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:44:57.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:45:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:45:27.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:45:27.847 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-04-01T05:45:27.847+0000 7f4548999640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 562581) UID: 0 2024-04-01T05:45:27.848 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-04-01T05:45:27.847+0000 7fd22327d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 562581) UID: 0 2024-04-01T05:45:27.848 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-04-01T05:45:27.847+0000 7ff29b736640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 562581) UID: 0 2024-04-01T05:45:27.848 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-04-01T05:45:27.847+0000 7f11fc45b640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 562581) UID: 0 2024-04-01T05:48:24.178 INFO:teuthology.orchestra.run.smithi064.stdout:generic/736 263s 2024-04-01T05:48:24.194 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/736 2024-04-01T05:48:24.194 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:48:24.197 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:24.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:48:24.224 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/737) 2024-04-01T05:48:25.126 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:48:25.137 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:48:25.137 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:25.499 INFO:teuthology.orchestra.run.smithi064.stdout:generic/737 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:48:25.511 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/737 2024-04-01T05:48:25.511 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/737 2024-04-01T05:48:25.511 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:48:25.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:25.548 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:48:25.548 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/738) 2024-04-01T05:48:26.162 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:48:26.183 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:48:26.183 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:generic/738 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:48:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/738 2024-04-01T05:48:26.612 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/738 2024-04-01T05:48:26.612 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:48:26.618 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:26.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:48:26.640 DEBUG:teuthology.orchestra.run.smithi064:> (cd /tmp/tmp.ALJJcUYdNQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/739) 2024-04-01T05:48:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:FSTYP -- ceph-fuse 2024-04-01T05:48:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM -- Linux/x86_64 smithi064 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-01T05:48:27.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:27.769 INFO:teuthology.orchestra.run.smithi064.stdout:generic/739 [not run] this test requires a valid $SCRATCH_DEV 2024-04-01T05:48:27.774 INFO:teuthology.orchestra.run.smithi064.stdout:Ran: generic/739 2024-04-01T05:48:27.775 INFO:teuthology.orchestra.run.smithi064.stdout:Not run: generic/739 2024-04-01T05:48:27.775 INFO:teuthology.orchestra.run.smithi064.stdout:Passed all 1 tests 2024-04-01T05:48:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:27.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-01T05:48:27.789 INFO:tasks.cephfs.xfstests_dev:=========================================================Total number of tests failed = 16========================================================= 2024-04-01T05:48:27.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo userdel --force --remove fsgqa 2024-04-01T05:48:29.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo userdel --force --remove 123456-fsgqa 2024-04-01T05:48:30.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo groupdel fsgqa 2024-04-01T05:48:30.286 INFO:teuthology.orchestra.run.smithi064.stderr:groupdel: group 'fsgqa' does not exist 2024-04-01T05:48:30.288 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-01T05:48:30.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:30.289 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.ALJJcUYdNQxfstests-dev/results) 2024-04-01T05:48:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-01T05:48:30.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T05:48:30.333 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.ALJJcUYdNQxfstests-dev 2024-04-01T05:48:30.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf /tmp/tmp.ALJJcUYdNQxfstests-dev 2024-04-01T05:48:30.969 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-01T05:48:30.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-01T05:48:31.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-01T05:48:31.314 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-01T05:48:31.314 INFO:tasks.cephfs.mount:Terminating background process 2024-04-01T05:48:31.314 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T05:48:31.314 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:31.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056374,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-01T05:48:31.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:48:31.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:31.476 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-01T05:48:31.641 INFO:tasks.cephfs.mount:Terminating background process 2024-04-01T05:48:31.641 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T05:48:31.641 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:31.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3299 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-01T05:48:31.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:48:31.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:31.678 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-01T05:48:31.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T05:48:31.739 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-01T05:48:31.740 INFO:tasks.cephfs.mount:Terminating background process 2024-04-01T05:48:31.740 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T05:48:31.740 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:31.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056374,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-01T05:48:31.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:48:31.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:31.786 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-01T05:48:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-01T05:48:31.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T05:48:31.871 INFO:tasks.cephfs.mount:Terminating background process 2024-04-01T05:48:31.872 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T05:48:31.872 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:31.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3299 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-01T05:48:31.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:48:31.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:31.888 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-01T05:48:31.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T05:48:31.944 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-01T05:48:31.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T05:48:33.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.655+0000 7fdffd222640 1 Processor -- start 2024-04-01T05:48:33.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.655+0000 7fdffd222640 1 -- start start 2024-04-01T05:48:33.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.657+0000 7fdffd222640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 0x7fdff815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.657+0000 7fdffd222640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 0x7fdff8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.658+0000 7fdffd222640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff815f2b0 con 0x7fdff80719e0 2024-04-01T05:48:33.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.658+0000 7fdff77fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdff80719e0 0x7fdff815e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:37268/0 (socket says 172.21.15.64:37268) 2024-04-01T05:48:33.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.658+0000 7fdff77fe640 1 -- 172.21.15.64:0/2808608266 learned_addr learned my addr 172.21.15.64:0/2808608266 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:33.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.658+0000 7fdff7fff640 1 --2- 172.21.15.64:0/2808608266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 0x7fdff815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.659+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdff80ce930 con 0x7fdff815ede0 2024-04-01T05:48:33.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.659+0000 7fdffca21640 1 --2- 172.21.15.64:0/2808608266 >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 0x7fdff8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.659+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdff815b840 con 0x7fdff8155130 2024-04-01T05:48:33.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.660+0000 7fdff7fff640 1 -- 172.21.15.64:0/2808608266 >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 msgr2=0x7fdff8071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:33.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.660+0000 7fdff7fff640 1 --2- 172.21.15.64:0/2808608266 >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 0x7fdff8071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.660+0000 7fdff7fff640 1 -- 172.21.15.64:0/2808608266 >> v1:172.21.15.100:6789/0 conn(0x7fdff80719e0 legacy=0x7fdff815e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:33.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.660+0000 7fdff7fff640 1 -- 172.21.15.64:0/2808608266 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff815b1f0 con 0x7fdff8155130 2024-04-01T05:48:33.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.660+0000 7fdffca21640 1 --2- 172.21.15.64:0/2808608266 >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 0x7fdff8071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.665+0000 7fdff7fff640 1 --2- 172.21.15.64:0/2808608266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 0x7fdff815bcd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80097a0 tx=0x7fdfe802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efc60fc7a96961b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:33.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.666+0000 7fdff6ffd640 1 -- 172.21.15.64:0/2808608266 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8035020 con 0x7fdff8155130 2024-04-01T05:48:33.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.666+0000 7fdff6ffd640 1 -- 172.21.15.64:0/2808608266 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe8031810 con 0x7fdff8155130 2024-04-01T05:48:33.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.666+0000 7fdff6ffd640 1 -- 172.21.15.64:0/2808608266 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8031be0 con 0x7fdff8155130 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 msgr2=0x7fdff815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 --2- 172.21.15.64:0/2808608266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 0x7fdff815bcd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80097a0 tx=0x7fdfe802b8b0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 shutdown_connections 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 --2- 172.21.15.64:0/2808608266 >> v2:172.21.15.100:3300/0 conn(0x7fdff815ede0 0x7fdff8071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 --2- 172.21.15.64:0/2808608266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155130 0x7fdff815bcd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 >> 172.21.15.64:0/2808608266 conn(0x7fdff80b3420 msgr2=0x7fdff80d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 shutdown_connections 2024-04-01T05:48:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.667+0000 7fdffd222640 1 -- 172.21.15.64:0/2808608266 wait complete. 2024-04-01T05:48:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.669+0000 7fdffd222640 1 Processor -- start 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.669+0000 7fdffd222640 1 -- start start 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.670+0000 7fdffd222640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 0x7fdff81512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.670+0000 7fdffd222640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 0x7fdff814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.670+0000 7fdffd222640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 0x7fdff8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffd222640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8155510 con 0x7fdff8155130 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffd222640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdff815f590 con 0x7fdff815ede0 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffd222640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdff806b3f0 con 0x7fdff8155690 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffca21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 0x7fdff8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffca21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 0x7fdff8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:48984/0 (socket says 172.21.15.64:48984) 2024-04-01T05:48:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdffca21640 1 -- 172.21.15.64:0/894392051 learned_addr learned my addr 172.21.15.64:0/894392051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.671+0000 7fdff77fe640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 0x7fdff814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.672+0000 7fdff7fff640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 0x7fdff81512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.672+0000 7fdff77fe640 1 -- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 msgr2=0x7fdff8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.672+0000 7fdff77fe640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 0x7fdff8156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.673+0000 7fdff77fe640 1 -- 172.21.15.64:0/894392051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 msgr2=0x7fdff81512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.673+0000 7fdff77fe640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 0x7fdff81512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:33.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.673+0000 7fdff77fe640 1 -- 172.21.15.64:0/894392051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff8151830 con 0x7fdff815ede0 2024-04-01T05:48:33.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.674+0000 7fdff7fff640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 0x7fdff81512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T05:48:33.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.674+0000 7fdff77fe640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 0x7fdff814de00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000f60 tx=0x7fdfec00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:33.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.684+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec017070 con 0x7fdff815ede0 2024-04-01T05:48:33.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.684+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfec005ce0 con 0x7fdff815ede0 2024-04-01T05:48:33.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.684+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0072d0 con 0x7fdff815ede0 2024-04-01T05:48:33.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.684+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff814e340 con 0x7fdff815ede0 2024-04-01T05:48:33.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.685+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff815c640 con 0x7fdff815ede0 2024-04-01T05:48:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.695+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fdfec01a070 con 0x7fdff815ede0 2024-04-01T05:48:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.697+0000 7fdff4ff9640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 0x7fdfe406df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.698+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdfec089a80 con 0x7fdff815ede0 2024-04-01T05:48:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.698+0000 7fdff7fff640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 0x7fdfe406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:33.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.704+0000 7fdff7fff640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 0x7fdfe406df20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8036000 tx=0x7fdfe8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:33.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.704+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff804f4b0 con 0x7fdff815ede0 2024-04-01T05:48:33.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:33.718+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fdff8151830 con 0x7fdff815ede0 2024-04-01T05:48:34.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.886+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdff815ae30 con 0x7fdff815ede0 2024-04-01T05:48:34.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.898+0000 7fdff4ff9640 1 -- 172.21.15.64:0/894392051 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7fdfec058020 con 0x7fdff815ede0 2024-04-01T05:48:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:34.899 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T05:48:34.900 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.901+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 msgr2=0x7fdfe406df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.901+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 0x7fdfe406df20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8036000 tx=0x7fdfe8008030 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.901+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 msgr2=0x7fdff814de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.901+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 0x7fdff814de00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000f60 tx=0x7fdfec00c800 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 shutdown_connections 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fdfe406ba60 0x7fdfe406df20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8155690 0x7fdff8156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff815ede0 0x7fdff814de00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 --2- 172.21.15.64:0/894392051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff8155130 0x7fdff81512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.902+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 >> 172.21.15.64:0/894392051 conn(0x7fdff80719e0 msgr2=0x7fdff806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:34.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.903+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 shutdown_connections 2024-04-01T05:48:34.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:34.903+0000 7fdffd222640 1 -- 172.21.15.64:0/894392051 wait complete. 2024-04-01T05:48:34.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T05:48:35.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.239+0000 7f0be81b5640 1 Processor -- start 2024-04-01T05:48:35.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.240+0000 7f0be81b5640 1 -- start start 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.242+0000 7f0be81b5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0be00cfdb0 0x7f0be00ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.242+0000 7f0be81b5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 0x7f0be0161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be81b5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be00b4bf0 con 0x7f0be0162140 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be81b5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f0be00b3750 con 0x7f0be00ae6d0 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be81b5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0be0162a50 con 0x7f0be00cfdb0 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be6752640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0be0162140 0x7f0be015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35452/0 (socket says 172.21.15.64:35452) 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be6752640 1 -- 172.21.15.64:0/952365651 learned_addr learned my addr 172.21.15.64:0/952365651 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5f51640 1 --2- 172.21.15.64:0/952365651 >> v2:172.21.15.100:3300/0 conn(0x7f0be00cfdb0 0x7f0be00ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5750640 1 --2- 172.21.15.64:0/952365651 >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 0x7f0be0161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be4f4f640 1 -- 172.21.15.64:0/952365651 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474488604 0 0) 0x7f0be00b4bf0 con 0x7f0be0162140 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5f51640 1 -- 172.21.15.64:0/952365651 >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 msgr2=0x7f0be0161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:35.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5f51640 1 --2- 172.21.15.64:0/952365651 >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 0x7f0be0161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5f51640 1 -- 172.21.15.64:0/952365651 >> v1:172.21.15.64:6789/0 conn(0x7f0be0162140 legacy=0x7f0be015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:35.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5f51640 1 -- 172.21.15.64:0/952365651 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be00b4e90 con 0x7f0be00cfdb0 2024-04-01T05:48:35.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.243+0000 7f0be5750640 1 --2- 172.21.15.64:0/952365651 >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 0x7f0be0161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:35.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.245+0000 7f0be5f51640 1 --2- 172.21.15.64:0/952365651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00ae190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0be00b4dc0 tx=0x7f0bdc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd8edb8f65b2744 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:35.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.246+0000 7f0be4f4f640 1 -- 172.21.15.64:0/952365651 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0315a0 con 0x7f0be00cfdb0 2024-04-01T05:48:35.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.246+0000 7f0be4f4f640 1 -- 172.21.15.64:0/952365651 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bdc037070 con 0x7f0be00cfdb0 2024-04-01T05:48:35.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.246+0000 7f0be4f4f640 1 -- 172.21.15.64:0/952365651 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc031960 con 0x7f0be00cfdb0 2024-04-01T05:48:35.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 -- 172.21.15.64:0/952365651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 msgr2=0x7f0be00ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:35.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 --2- 172.21.15.64:0/952365651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00ae190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0be00b4dc0 tx=0x7f0bdc02b840 comp rx=0 tx=0).stop 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 -- 172.21.15.64:0/952365651 shutdown_connections 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 --2- 172.21.15.64:0/952365651 >> v2:172.21.15.64:3300/0 conn(0x7f0be00ae6d0 0x7f0be0161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 --2- 172.21.15.64:0/952365651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00ae190 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.247+0000 7f0be81b5640 1 -- 172.21.15.64:0/952365651 >> 172.21.15.64:0/952365651 conn(0x7f0be00d0450 msgr2=0x7f0be00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.248+0000 7f0be81b5640 1 -- 172.21.15.64:0/952365651 shutdown_connections 2024-04-01T05:48:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.248+0000 7f0be81b5640 1 -- 172.21.15.64:0/952365651 wait complete. 2024-04-01T05:48:35.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.253+0000 7f0be81b5640 1 Processor -- start 2024-04-01T05:48:35.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.253+0000 7f0be81b5640 1 -- start start 2024-04-01T05:48:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be00ae6d0 0x7f0be0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0be0162140 0x7f0be0066e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0072200 con 0x7f0be00ae6d0 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0071f20 con 0x7f0be00cfdb0 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.259+0000 7f0be81b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00724e0 con 0x7f0be0162140 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:50852/0 (socket says 172.21.15.64:50852) 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 -- 172.21.15.64:0/2180306545 learned_addr learned my addr 172.21.15.64:0/2180306545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 -- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0be0162140 msgr2=0x7f0be0066e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0be0162140 0x7f0be0066e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 -- 172.21.15.64:0/2180306545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be00ae6d0 msgr2=0x7f0be0155fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be00ae6d0 0x7f0be0155fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.262+0000 7f0be5750640 1 -- 172.21.15.64:0/2180306545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be015b2f0 con 0x7f0be00cfdb0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.263+0000 7f0be5750640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e960 tx=0x7f0bd000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.263+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0017070 con 0x7f0be00cfdb0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.264+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd000ca00 con 0x7f0be00cfdb0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.264+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000cdb0 con 0x7f0be00cfdb0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.264+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be015b5d0 con 0x7f0be00cfdb0 2024-04-01T05:48:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.264+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be015bb90 con 0x7f0be00cfdb0 2024-04-01T05:48:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.266+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0bd001a070 con 0x7f0be00cfdb0 2024-04-01T05:48:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.266+0000 7f0bd67fc640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 0x7f0bc406dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.266+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd008a980 con 0x7f0be00cfdb0 2024-04-01T05:48:35.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.267+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be00688b0 con 0x7f0be00cfdb0 2024-04-01T05:48:35.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.267+0000 7f0be5f51640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 0x7f0bc406dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:35.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.268+0000 7f0be5f51640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 0x7f0bc406dea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc036000 tx=0x7f0bdc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:35.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.272+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0bd0059700 con 0x7f0be00cfdb0 2024-04-01T05:48:35.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.688+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0be0157720 con 0x7f0be00cfdb0 2024-04-01T05:48:35.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.691+0000 7f0bd67fc640 1 -- 172.21.15.64:0/2180306545 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16645 (secure 0 0 0) 0x7f0bd0059000 con 0x7f0be00cfdb0 2024-04-01T05:48:35.693 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:35.693 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:46.560584+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T05:48:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 msgr2=0x7f0bc406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 0x7f0bc406dea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc036000 tx=0x7f0bdc008030 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 msgr2=0x7f0be00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e960 tx=0x7f0bd000ee20 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 shutdown_connections 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0bc406b9e0 0x7f0bc406dea0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0be0162140 0x7f0be0066e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be00cfdb0 0x7f0be00cefc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 --2- 172.21.15.64:0/2180306545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be00ae6d0 0x7f0be0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.697+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 >> 172.21.15.64:0/2180306545 conn(0x7f0be00d0450 msgr2=0x7f0be0161090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:35.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.698+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 shutdown_connections 2024-04-01T05:48:35.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:35.698+0000 7f0be81b5640 1 -- 172.21.15.64:0/2180306545 wait complete. 2024-04-01T05:48:35.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-01T00:13:34.778476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-01T00:14:43.503546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-01T00:14:43.889938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-01T05:48:35.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-01T05:48:36.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.163+0000 7fec24e30640 1 Processor -- start 2024-04-01T05:48:36.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.163+0000 7fec24e30640 1 -- start start 2024-04-01T05:48:36.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec24e30640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec2015e400 0x7fec20154720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec24e30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 0x7fec200b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec24e30640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200718d0 con 0x7fec2006ece0 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec24e30640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fec2006d580 con 0x7fec2015e400 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec24e30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec200b3a70 con 0x7fec20071460 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec1dd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec2015e400 0x7fec20154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec1e59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fec2006ece0 0x7fec2015dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:37284/0 (socket says 172.21.15.64:37284) 2024-04-01T05:48:36.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec1e59c640 1 -- 172.21.15.64:0/1258621808 learned_addr learned my addr 172.21.15.64:0/1258621808 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.165+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/1258621808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 0x7fec200b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1d59a640 1 -- 172.21.15.64:0/1258621808 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769772152 0 0) 0x7fec200718d0 con 0x7fec2006ece0 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1d59a640 1 -- 172.21.15.64:0/1258621808 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec08003520 con 0x7fec2006ece0 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1ed9d640 1 -- 172.21.15.64:0/1258621808 >> v2:172.21.15.64:3300/0 conn(0x7fec2015e400 msgr2=0x7fec20154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/1258621808 >> v2:172.21.15.64:3300/0 conn(0x7fec2015e400 0x7fec20154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1ed9d640 1 -- 172.21.15.64:0/1258621808 >> v1:172.21.15.100:6789/0 conn(0x7fec2006ece0 legacy=0x7fec2015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:36.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.166+0000 7fec1ed9d640 1 -- 172.21.15.64:0/1258621808 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2006cc50 con 0x7fec20071460 2024-04-01T05:48:36.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.170+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/1258621808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 0x7fec200b2cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fec140097a0 tx=0x7fec1402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610c798befcf309d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:36.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.170+0000 7fec1d59a640 1 -- 172.21.15.64:0/1258621808 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14035020 con 0x7fec20071460 2024-04-01T05:48:36.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.170+0000 7fec1d59a640 1 -- 172.21.15.64:0/1258621808 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec14031830 con 0x7fec20071460 2024-04-01T05:48:36.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.170+0000 7fec1d59a640 1 -- 172.21.15.64:0/1258621808 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14031be0 con 0x7fec20071460 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.171+0000 7fec24e30640 1 -- 172.21.15.64:0/1258621808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 msgr2=0x7fec200b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.171+0000 7fec24e30640 1 --2- 172.21.15.64:0/1258621808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 0x7fec200b2cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fec140097a0 tx=0x7fec1402b8d0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.172+0000 7fec24e30640 1 -- 172.21.15.64:0/1258621808 shutdown_connections 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.172+0000 7fec24e30640 1 --2- 172.21.15.64:0/1258621808 >> v2:172.21.15.64:3300/0 conn(0x7fec2015e400 0x7fec20154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.172+0000 7fec24e30640 1 --2- 172.21.15.64:0/1258621808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec20071460 0x7fec200b2cf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.172+0000 7fec24e30640 1 -- 172.21.15.64:0/1258621808 >> 172.21.15.64:0/1258621808 conn(0x7fec200b1730 msgr2=0x7fec200b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.175+0000 7fec24e30640 1 -- 172.21.15.64:0/1258621808 shutdown_connections 2024-04-01T05:48:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.180+0000 7fec24e30640 1 -- 172.21.15.64:0/1258621808 wait complete. 2024-04-01T05:48:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.185+0000 7fec24e30640 1 Processor -- start 2024-04-01T05:48:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.185+0000 7fec24e30640 1 -- start start 2024-04-01T05:48:36.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.189+0000 7fec24e30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2006ece0 0x7fec20158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.189+0000 7fec24e30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec20071460 0x7fec201453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.189+0000 7fec24e30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec1ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:36.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec1dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec20071460 0x7fec201453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:36.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec1ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:50864/0 (socket says 172.21.15.64:50864) 2024-04-01T05:48:36.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec1ed9d640 1 -- 172.21.15.64:0/2002885115 learned_addr learned my addr 172.21.15.64:0/2002885115 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec24e30640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec200b3230 con 0x7fec20071460 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec2006d210 con 0x7fec2015e400 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.190+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec200b33b0 con 0x7fec2006ece0 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.191+0000 7fec1ed9d640 1 -- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2006ece0 msgr2=0x7fec20158780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.191+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2006ece0 0x7fec20158780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.191+0000 7fec1ed9d640 1 -- 172.21.15.64:0/2002885115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec20071460 msgr2=0x7fec201453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.191+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec20071460 0x7fec201453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.191+0000 7fec1ed9d640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20159000 con 0x7fec2015e400 2024-04-01T05:48:36.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.192+0000 7fec1ed9d640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fec140098c0 tx=0x7fec140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.193+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14035040 con 0x7fec2015e400 2024-04-01T05:48:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.193+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec14003610 con 0x7fec2015e400 2024-04-01T05:48:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.193+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14003a00 con 0x7fec2015e400 2024-04-01T05:48:36.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.193+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec201592e0 con 0x7fec2015e400 2024-04-01T05:48:36.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.194+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201547c0 con 0x7fec2015e400 2024-04-01T05:48:36.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.199+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fec14038020 con 0x7fec2015e400 2024-04-01T05:48:36.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.200+0000 7fec0effd640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 0x7fec0006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:36.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.201+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fec14078d70 con 0x7fec2015e400 2024-04-01T05:48:36.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.203+0000 7fec1e59c640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 0x7fec0006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:36.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.205+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04005290 con 0x7fec2015e400 2024-04-01T05:48:36.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.206+0000 7fec1e59c640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 0x7fec0006dea0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fec180012e0 tx=0x7fec180016d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:36.213+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fec1407b3d0 con 0x7fec2015e400 2024-04-01T05:48:37.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.204+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec04005850 con 0x7fec2015e400 2024-04-01T05:48:37.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.209+0000 7fec0effd640 1 -- 172.21.15.64:0/2002885115 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fec1407acd0 con 0x7fec2015e400 2024-04-01T05:48:37.210 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-04-01T05:48:37.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.215+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 msgr2=0x7fec0006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:37.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.215+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 0x7fec0006dea0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fec180012e0 tx=0x7fec180016d0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.215+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 msgr2=0x7fec2015ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:37.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.215+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fec140098c0 tx=0x7fec140046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 shutdown_connections 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fec0006b9e0 0x7fec0006dea0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015e400 0x7fec2015ab00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec20071460 0x7fec201453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 --2- 172.21.15.64:0/2002885115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2006ece0 0x7fec20158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.216+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 >> 172.21.15.64:0/2002885115 conn(0x7fec200b1730 msgr2=0x7fec201539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.217+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 shutdown_connections 2024-04-01T05:48:37.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.217+0000 7fec24e30640 1 -- 172.21.15.64:0/2002885115 wait complete. 2024-04-01T05:48:37.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-01T05:48:37.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.629+0000 7f06c3a30640 1 Processor -- start 2024-04-01T05:48:37.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.629+0000 7f06c3a30640 1 -- start start 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c3a30640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06bc0cac70 0x7f06bc071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c3a30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c3a30640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06bc0ca9e0 con 0x7f06bc15df80 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c3a30640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f06bc071c60 con 0x7f06bc0cac70 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c3a30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06bc071eb0 con 0x7f06bc15eb10 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c1fcd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c1fcd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:48996/0 (socket says 172.21.15.64:48996) 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.633+0000 7f06c1fcd640 1 -- 172.21.15.64:0/1164183148 learned_addr learned my addr 172.21.15.64:0/1164183148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.634+0000 7f06c1fcd640 1 -- 172.21.15.64:0/1164183148 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0cac70 msgr2=0x7f06bc071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.634+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/1164183148 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0cac70 0x7f06bc071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.634+0000 7f06c1fcd640 1 -- 172.21.15.64:0/1164183148 >> v1:172.21.15.64:6789/0 conn(0x7f06bc15df80 legacy=0x7f06bc15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.634+0000 7f06c1fcd640 1 -- 172.21.15.64:0/1164183148 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc072190 con 0x7f06bc15eb10 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.635+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/1164183148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ffd0 tx=0x7f06b403cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ef2e4b49d76b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.635+0000 7f06b3fff640 1 -- 172.21.15.64:0/1164183148 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4003b90 con 0x7f06bc15eb10 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.636+0000 7f06b3fff640 1 -- 172.21.15.64:0/1164183148 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4003d30 con 0x7f06bc15eb10 2024-04-01T05:48:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.636+0000 7f06b3fff640 1 -- 172.21.15.64:0/1164183148 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b403da20 con 0x7f06bc15eb10 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 -- 172.21.15.64:0/1164183148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 msgr2=0x7f06bc0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 --2- 172.21.15.64:0/1164183148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ffd0 tx=0x7f06b403cc70 comp rx=0 tx=0).stop 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 -- 172.21.15.64:0/1164183148 shutdown_connections 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 --2- 172.21.15.64:0/1164183148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15eb10 0x7f06bc0cd940 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 --2- 172.21.15.64:0/1164183148 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0cac70 0x7f06bc071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.638+0000 7f06c3a30640 1 -- 172.21.15.64:0/1164183148 >> 172.21.15.64:0/1164183148 conn(0x7f06bc0cfa90 msgr2=0x7f06bc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.642+0000 7f06c3a30640 1 -- 172.21.15.64:0/1164183148 shutdown_connections 2024-04-01T05:48:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.649+0000 7f06c3a30640 1 -- 172.21.15.64:0/1164183148 wait complete. 2024-04-01T05:48:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.657+0000 7f06c3a30640 1 Processor -- start 2024-04-01T05:48:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.659+0000 7f06c3a30640 1 -- start start 2024-04-01T05:48:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.663+0000 7f06c3a30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.663+0000 7f06c3a30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15df80 0x7f06bc157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c3a30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 0x7f06bc157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c17cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c17cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59388/0 (socket says 172.21.15.64:59388) 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc072a30 con 0x7f06bc0cac70 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c17cc640 1 -- 172.21.15.64:0/2372832250 learned_addr learned my addr 172.21.15.64:0/2372832250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc072750 con 0x7f06bc15eb10 2024-04-01T05:48:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06bc15a6d0 con 0x7f06bc15df80 2024-04-01T05:48:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 0x7f06bc157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.664+0000 7f06c0fcb640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15df80 0x7f06bc157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.665+0000 7f06c1fcd640 1 -- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15df80 msgr2=0x7f06bc157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:37.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.665+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15df80 0x7f06bc157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.665+0000 7f06c1fcd640 1 -- 172.21.15.64:0/2372832250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 msgr2=0x7f06bc1484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:37.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.665+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.665+0000 7f06c1fcd640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc0c7970 con 0x7f06bc15eb10 2024-04-01T05:48:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.666+0000 7f06c1fcd640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 0x7f06bc157840 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f06b4004000 tx=0x7f06b4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.666+0000 7f06c17cc640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-01T05:48:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.667+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4055030 con 0x7f06bc15eb10 2024-04-01T05:48:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.667+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b403de40 con 0x7f06bc15eb10 2024-04-01T05:48:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.667+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc0c7c50 con 0x7f06bc15eb10 2024-04-01T05:48:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.668+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400fa60 con 0x7f06bc15eb10 2024-04-01T05:48:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.668+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc0c8210 con 0x7f06bc15eb10 2024-04-01T05:48:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.671+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06bc0681a0 con 0x7f06bc15eb10 2024-04-01T05:48:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.673+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f06b4058070 con 0x7f06bc15eb10 2024-04-01T05:48:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.674+0000 7f06b1ffb640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 0x7f06a006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.674+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f06b40bde70 con 0x7f06bc15eb10 2024-04-01T05:48:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.674+0000 7f06c17cc640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 0x7f06a006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.677+0000 7f06c17cc640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 0x7f06a006df20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:37.681+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f06b40593b0 con 0x7f06bc15eb10 2024-04-01T05:48:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.016+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06bc0c6660 con 0x7f06bc15eb10 2024-04-01T05:48:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.017+0000 7f06b1ffb640 1 -- 172.21.15.64:0/2372832250 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f06b408caf0 con 0x7f06bc15eb10 2024-04-01T05:48:38.020 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-04-01T05:48:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.020+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 msgr2=0x7f06a006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.020+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 0x7f06a006df20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).stop 2024-04-01T05:48:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.020+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 msgr2=0x7f06bc157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.020+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 0x7f06bc157840 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f06b4004000 tx=0x7f06b4004a40 comp rx=0 tx=0).stop 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 shutdown_connections 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f06a006ba60 0x7f06a006df20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06bc15eb10 0x7f06bc157840 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06bc15df80 0x7f06bc157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 --2- 172.21.15.64:0/2372832250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc0cac70 0x7f06bc1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 >> 172.21.15.64:0/2372832250 conn(0x7f06bc0cfa90 msgr2=0x7f06bc1582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 shutdown_connections 2024-04-01T05:48:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.023+0000 7f06c3a30640 1 -- 172.21.15.64:0/2372832250 wait complete. 2024-04-01T05:48:38.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-01T05:48:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.465+0000 7f0058e5e640 1 Processor -- start 2024-04-01T05:48:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.466+0000 7f0058e5e640 1 -- start start 2024-04-01T05:48:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0058e5e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0054143810 0x7f0054071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0058e5e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406bbb0 0x7f005415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0058e5e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0054070dc0 con 0x7f0054072380 2024-04-01T05:48:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0058e5e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0054071080 con 0x7f0054143810 2024-04-01T05:48:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0058e5e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0054071360 con 0x7f005406bbb0 2024-04-01T05:48:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.470+0000 7f0053fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0054072380 0x7f00540cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:37300/0 (socket says 172.21.15.64:37300) 2024-04-01T05:48:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.471+0000 7f0053fff640 1 -- 172.21.15.64:0/1545401420 learned_addr learned my addr 172.21.15.64:0/1545401420 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.471+0000 7f0052ffd640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406bbb0 0x7f005415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.471+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949681143 0 0) 0x7f0054070dc0 con 0x7f0054072380 2024-04-01T05:48:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.471+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f003c003520 con 0x7f0054072380 2024-04-01T05:48:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.471+0000 7f00537fe640 1 --2- 172.21.15.64:0/1545401420 >> v2:172.21.15.100:3300/0 conn(0x7f0054143810 0x7f0054071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 574897117 0 0) 0x7f003c003520 con 0x7f0054072380 2024-04-01T05:48:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406bbb0 msgr2=0x7f005415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f005406bbb0 0x7f005415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 >> v2:172.21.15.100:3300/0 conn(0x7f0054143810 msgr2=0x7f0054071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 --2- 172.21.15.64:0/1545401420 >> v2:172.21.15.100:3300/0 conn(0x7f0054143810 0x7f0054071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0054071640 con 0x7f0054072380 2024-04-01T05:48:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f004c0019f0 con 0x7f0054072380 2024-04-01T05:48:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 >> v1:172.21.15.100:6789/0 conn(0x7f0054072380 legacy=0x7f00540cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003c003a20 0x7f003c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c004800 0x7f003c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f003c0051c0 0x7f003c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0054071080 con 0x7f003c003a20 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0054071360 con 0x7f003c004800 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f003c00bfb0 con 0x7f003c0051c0 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.472+0000 7f00537fe640 1 --2- 172.21.15.64:0/1545401420 >> v2:172.21.15.100:3300/0 conn(0x7f0054143810 0x7f0054071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.473+0000 7f00537fe640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003c003a20 0x7f003c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.473+0000 7f0053fff640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f003c0051c0 0x7f003c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0052ffd640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c004800 0x7f003c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0053fff640 1 -- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c004800 msgr2=0x7f003c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0053fff640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c004800 0x7f003c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0053fff640 1 -- 172.21.15.64:0/1545401420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003c003a20 msgr2=0x7f003c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0053fff640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003c003a20 0x7f003c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.474+0000 7f0053fff640 1 -- 172.21.15.64:0/1545401420 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c003520 con 0x7f003c0051c0 2024-04-01T05:48:38.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.475+0000 7f00537fe640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003c003a20 0x7f003c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:38.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.478+0000 7f0053fff640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f003c0051c0 0x7f003c00b9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f004c00a870 tx=0x7f003c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae68d35a0e88564 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.478+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c00a4c0 con 0x7f003c0051c0 2024-04-01T05:48:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.479+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004c034650 con 0x7f003c0051c0 2024-04-01T05:48:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.479+0000 7f00527fc640 1 -- 172.21.15.64:0/1545401420 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c034a40 con 0x7f003c0051c0 2024-04-01T05:48:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.481+0000 7f0058e5e640 1 -- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f003c0051c0 msgr2=0x7f003c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.481+0000 7f0058e5e640 1 --2- 172.21.15.64:0/1545401420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f003c0051c0 0x7f003c00b9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f004c00a870 tx=0x7f003c005830 comp rx=0 tx=0).stop 2024-04-01T05:48:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.481+0000 7f0053fff640 1 -- 172.21.15.64:0/1545401420 reap_dead start 2024-04-01T05:48:38.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.482+0000 7f0058e5e640 1 -- 172.21.15.64:0/1545401420 shutdown_connections 2024-04-01T05:48:38.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.482+0000 7f0058e5e640 1 -- 172.21.15.64:0/1545401420 >> 172.21.15.64:0/1545401420 conn(0x7f005406b0c0 msgr2=0x7f005406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:38.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.482+0000 7f0058e5e640 1 -- 172.21.15.64:0/1545401420 shutdown_connections 2024-04-01T05:48:38.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.482+0000 7f0058e5e640 1 -- 172.21.15.64:0/1545401420 wait complete. 2024-04-01T05:48:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.493+0000 7f0058e5e640 1 Processor -- start 2024-04-01T05:48:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.494+0000 7f0058e5e640 1 -- start start 2024-04-01T05:48:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f0058e5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f0058e5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 0x7f00541517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f0058e5e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cc790 0x7f0054153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f00537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f00537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:50878/0 (socket says 172.21.15.64:50878) 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f0052ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 0x7f00541517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.499+0000 7f0052ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 0x7f00541517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59410/0 (socket says 172.21.15.64:59410) 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.500+0000 7f00537fe640 1 -- 172.21.15.64:0/2651572973 learned_addr learned my addr 172.21.15.64:0/2651572973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.500+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00540d0560 con 0x7f00540cc310 2024-04-01T05:48:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.500+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00540d03e0 con 0x7f003c0051c0 2024-04-01T05:48:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.500+0000 7f0053fff640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cc790 0x7f0054153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.500+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f005415b5a0 con 0x7f00540cc790 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.501+0000 7f00537fe640 1 -- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cc790 msgr2=0x7f0054153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.501+0000 7f00537fe640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cc790 0x7f0054153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.501+0000 7f00537fe640 1 -- 172.21.15.64:0/2651572973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 msgr2=0x7f00541517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.501+0000 7f00537fe640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 0x7f00541517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.501+0000 7f00537fe640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0054152110 con 0x7f003c0051c0 2024-04-01T05:48:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.502+0000 7f00537fe640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0048011590 tx=0x7f0048011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.502+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048016480 con 0x7f003c0051c0 2024-04-01T05:48:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.502+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004801a030 con 0x7f003c0051c0 2024-04-01T05:48:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.502+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048016820 con 0x7f003c0051c0 2024-04-01T05:48:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.502+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005414f300 con 0x7f003c0051c0 2024-04-01T05:48:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.503+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005414f8c0 con 0x7f003c0051c0 2024-04-01T05:48:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.508+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0048015460 con 0x7f003c0051c0 2024-04-01T05:48:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.508+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005404f4b0 con 0x7f003c0051c0 2024-04-01T05:48:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.509+0000 7f003bfff640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 0x7f004006ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.509+0000 7f0052ffd640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 0x7f004006ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.510+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f004808b1c0 con 0x7f003c0051c0 2024-04-01T05:48:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.510+0000 7f0052ffd640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 0x7f004006ded0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f00541549c0 tx=0x7f0044008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.514+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f00480598f0 con 0x7f003c0051c0 2024-04-01T05:48:38.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.839+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f00541395f0 con 0x7f003c0051c0 2024-04-01T05:48:38.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.841+0000 7f003bfff640 1 -- 172.21.15.64:0/2651572973 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f004805d990 con 0x7f003c0051c0 2024-04-01T05:48:38.842 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 msgr2=0x7f004006ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 0x7f004006ded0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f00541549c0 tx=0x7f0044008040 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 msgr2=0x7f0054138660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0048011590 tx=0x7f0048011a50 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 shutdown_connections 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f004006ba10 0x7f004006ded0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00540cc790 0x7f0054153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00540cc310 0x7f00541517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 --2- 172.21.15.64:0/2651572973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c0051c0 0x7f0054138660 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.845+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 >> 172.21.15.64:0/2651572973 conn(0x7f005406b0c0 msgr2=0x7f00540b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.846+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 shutdown_connections 2024-04-01T05:48:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:38.846+0000 7f0058e5e640 1 -- 172.21.15.64:0/2651572973 wait complete. 2024-04-01T05:48:38.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T05:48:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.093+0000 7f962178a640 1 Processor -- start 2024-04-01T05:48:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.094+0000 7f962178a640 1 -- start start 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.094+0000 7f962178a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f961c0cff40 0x7f961c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.094+0000 7f962178a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c06d6a0 0x7f961c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f962178a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c0cf2e0 con 0x7f961c0ce120 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f962178a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f961c072ab0 con 0x7f961c0cff40 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f962178a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f961c072d50 con 0x7f961c06d6a0 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f961c0cff40 0x7f961c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f961c0cff40 0x7f961c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59412/0 (socket says 172.21.15.64:59412) 2024-04-01T05:48:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 -- 172.21.15.64:0/4288650552 learned_addr learned my addr 172.21.15.64:0/4288650552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961b7fe640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c06d6a0 0x7f961c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 -- 172.21.15.64:0/4288650552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c06d6a0 msgr2=0x7f961c06db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c06d6a0 0x7f961c06db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 -- 172.21.15.64:0/4288650552 >> v1:172.21.15.100:6789/0 conn(0x7f961c0ce120 legacy=0x7f961c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.095+0000 7f961affd640 1 -- 172.21.15.64:0/4288650552 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c06cbc0 con 0x7f961c0cff40 2024-04-01T05:48:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.097+0000 7f961affd640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c071ae0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f961402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cabd77ab8f951b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.097+0000 7f9619ffb640 1 -- 172.21.15.64:0/4288650552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961402cb00 con 0x7f961c0cff40 2024-04-01T05:48:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.097+0000 7f9619ffb640 1 -- 172.21.15.64:0/4288650552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961402cca0 con 0x7f961c0cff40 2024-04-01T05:48:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.097+0000 7f9619ffb640 1 -- 172.21.15.64:0/4288650552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614034610 con 0x7f961c0cff40 2024-04-01T05:48:39.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.098+0000 7f962178a640 1 -- 172.21.15.64:0/4288650552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 msgr2=0x7f961c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:39.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.098+0000 7f962178a640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c071ae0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f961402bf60 comp rx=0 tx=0).stop 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.098+0000 7f962178a640 1 -- 172.21.15.64:0/4288650552 shutdown_connections 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.098+0000 7f962178a640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c06d6a0 0x7f961c06db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.098+0000 7f962178a640 1 --2- 172.21.15.64:0/4288650552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c071ae0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.099+0000 7f962178a640 1 -- 172.21.15.64:0/4288650552 >> 172.21.15.64:0/4288650552 conn(0x7f961c06b0c0 msgr2=0x7f961c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.100+0000 7f962178a640 1 -- 172.21.15.64:0/4288650552 shutdown_connections 2024-04-01T05:48:39.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.100+0000 7f962178a640 1 -- 172.21.15.64:0/4288650552 wait complete. 2024-04-01T05:48:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.107+0000 7f962178a640 1 Processor -- start 2024-04-01T05:48:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.107+0000 7f962178a640 1 -- start start 2024-04-01T05:48:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f962178a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f962178a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 0x7f961c09a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f962178a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f961affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f961affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:50884/0 (socket says 172.21.15.64:50884) 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f961affd640 1 -- 172.21.15.64:0/2369631841 learned_addr learned my addr 172.21.15.64:0/2369631841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f961a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 0x7f961c09a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.111+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.112+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f961c15e3f0 con 0x7f961c0cff40 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.113+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f961c0b2af0 con 0x7f961c06d6a0 2024-04-01T05:48:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.113+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f961c15e570 con 0x7f961c0ce120 2024-04-01T05:48:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.115+0000 7f961b7fe640 1 -- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 msgr2=0x7f961c09a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.115+0000 7f961b7fe640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 0x7f961c09a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.115+0000 7f961b7fe640 1 -- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 msgr2=0x7f961c14c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.115+0000 7f961b7fe640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.116+0000 7f961b7fe640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c09b360 con 0x7f961c0cff40 2024-04-01T05:48:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.116+0000 7f961affd640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.116+0000 7f961a7fc640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 0x7f961c09a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:39.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.117+0000 7f961b7fe640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c158c50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f961000e7f0 tx=0x7f961000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.118+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610017070 con 0x7f961c0cff40 2024-04-01T05:48:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.118+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961000c940 con 0x7f961c0cff40 2024-04-01T05:48:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.118+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c09b640 con 0x7f961c0cff40 2024-04-01T05:48:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.118+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c09bc00 con 0x7f961c0cff40 2024-04-01T05:48:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.118+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961000cd10 con 0x7f961c0cff40 2024-04-01T05:48:39.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.123+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f961001a070 con 0x7f961c0cff40 2024-04-01T05:48:39.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.124+0000 7f96037fe640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 0x7f960806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:39.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.124+0000 7f961affd640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 0x7f960806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:39.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.124+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f961008a5e0 con 0x7f961c0cff40 2024-04-01T05:48:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.125+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc005290 con 0x7f961c0cff40 2024-04-01T05:48:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.125+0000 7f961affd640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 0x7f960806de50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f9614030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:39.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:39.141+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f9610058c10 con 0x7f961c0cff40 2024-04-01T05:48:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.370+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95fc005850 con 0x7f961c0cff40 2024-04-01T05:48:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.371+0000 7f96037fe640 1 -- 172.21.15.64:0/2369631841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7f961005ccb0 con 0x7f961c0cff40 2024-04-01T05:48:40.373 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:40.373 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T05:48:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 msgr2=0x7f960806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 0x7f960806de50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f9614030000 comp rx=0 tx=0).stop 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 msgr2=0x7f961c158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c158c50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f961000e7f0 tx=0x7f961000ecb0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 shutdown_connections 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f960806b990 0x7f960806de50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f961c0cff40 0x7f961c158c50 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f961c0ce120 0x7f961c09a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 --2- 172.21.15.64:0/2369631841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f961c06d6a0 0x7f961c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.378+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 >> 172.21.15.64:0/2369631841 conn(0x7f961c06b0c0 msgr2=0x7f961c15d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:40.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.382+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 shutdown_connections 2024-04-01T05:48:40.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.382+0000 7f962178a640 1 -- 172.21.15.64:0/2369631841 wait complete. 2024-04-01T05:48:40.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T05:48:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.703+0000 7ff155d02640 1 Processor -- start 2024-04-01T05:48:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.703+0000 7ff155d02640 1 -- start start 2024-04-01T05:48:40.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.707+0000 7ff155d02640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff1500d0100 0x7ff15008b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:40.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.707+0000 7ff155d02640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1500ccd10 con 0x7ff1500cc150 2024-04-01T05:48:40.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.707+0000 7ff155d02640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15016b0b0 con 0x7ff1500cfd20 2024-04-01T05:48:40.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.707+0000 7ff155d02640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff15008bca0 con 0x7ff1500d0100 2024-04-01T05:48:40.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.707+0000 7ff14f7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff1500cfd20 0x7ff15016f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:37314/0 (socket says 172.21.15.64:37314) 2024-04-01T05:48:40.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.708+0000 7ff14f7fe640 1 -- 172.21.15.64:0/1158625412 learned_addr learned my addr 172.21.15.64:0/1158625412 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:40.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.708+0000 7ff14effd640 1 --2- 172.21.15.64:0/1158625412 >> v2:172.21.15.100:3300/0 conn(0x7ff1500d0100 0x7ff15008b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:40.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.708+0000 7ff14e7fc640 1 -- 172.21.15.64:0/1158625412 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729317660 0 0) 0x7ff15016b0b0 con 0x7ff1500cfd20 2024-04-01T05:48:40.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.709+0000 7ff14effd640 1 -- 172.21.15.64:0/1158625412 >> v1:172.21.15.100:6789/0 conn(0x7ff1500cfd20 legacy=0x7ff15016f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.709+0000 7ff14effd640 1 -- 172.21.15.64:0/1158625412 >> v1:172.21.15.64:6789/0 conn(0x7ff1500cc150 legacy=0x7ff1500cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.709+0000 7ff14effd640 1 -- 172.21.15.64:0/1158625412 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15006e220 con 0x7ff1500d0100 2024-04-01T05:48:40.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.712+0000 7ff14effd640 1 --2- 172.21.15.64:0/1158625412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff15008b760 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff1400097a0 tx=0x7ff14002ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4476c90193332bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:40.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.712+0000 7ff14e7fc640 1 -- 172.21.15.64:0/1158625412 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14002e830 con 0x7ff1500d0100 2024-04-01T05:48:40.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.712+0000 7ff14e7fc640 1 -- 172.21.15.64:0/1158625412 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14002e9d0 con 0x7ff1500d0100 2024-04-01T05:48:40.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.712+0000 7ff14e7fc640 1 -- 172.21.15.64:0/1158625412 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14002eda0 con 0x7ff1500d0100 2024-04-01T05:48:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 -- 172.21.15.64:0/1158625412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 msgr2=0x7ff15008b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 --2- 172.21.15.64:0/1158625412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff15008b760 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff1400097a0 tx=0x7ff14002ca70 comp rx=0 tx=0).stop 2024-04-01T05:48:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 -- 172.21.15.64:0/1158625412 shutdown_connections 2024-04-01T05:48:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 --2- 172.21.15.64:0/1158625412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff15008b760 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 -- 172.21.15.64:0/1158625412 >> 172.21.15.64:0/1158625412 conn(0x7ff15006b0c0 msgr2=0x7ff15006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:40.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.715+0000 7ff155d02640 1 -- 172.21.15.64:0/1158625412 shutdown_connections 2024-04-01T05:48:40.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.716+0000 7ff155d02640 1 -- 172.21.15.64:0/1158625412 wait complete. 2024-04-01T05:48:40.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.722+0000 7ff155d02640 1 Processor -- start 2024-04-01T05:48:40.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.723+0000 7ff155d02640 1 -- start start 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.724+0000 7ff155d02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1500cc150 0x7ff15015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.724+0000 7ff155d02640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff155d02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff1502048d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff155d02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff15016ed10 con 0x7ff1500cc150 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff155d02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff15016ea30 con 0x7ff1500d0100 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff155d02640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff15008b3c0 con 0x7ff1500cfd20 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:49078/0 (socket says 172.21.15.64:49078) 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1500cc150 0x7ff15015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 -- 172.21.15.64:0/3014064783 learned_addr learned my addr 172.21.15.64:0/3014064783 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 -- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 msgr2=0x7ff1502048d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff1502048d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 -- 172.21.15.64:0/3014064783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1500cc150 msgr2=0x7ff15015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1500cc150 0x7ff15015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 -- 172.21.15.64:0/3014064783 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff150205110 con 0x7ff1500cfd20 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.725+0000 7ff14effd640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098c0 tx=0x7ff140004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.726+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140048070 con 0x7ff1500cfd20 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.726+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140038dd0 con 0x7ff1500cfd20 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.726+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14002f640 con 0x7ff1500cfd20 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.726+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15009ab90 con 0x7ff1500cfd20 2024-04-01T05:48:40.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.726+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15009b150 con 0x7ff1500cfd20 2024-04-01T05:48:40.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.729+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff14004c430 con 0x7ff1500cfd20 2024-04-01T05:48:40.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.730+0000 7ff137fff640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 0x7ff13c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:40.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.730+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff1400ac4c0 con 0x7ff1500cfd20 2024-04-01T05:48:40.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.730+0000 7ff14f7fe640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 0x7ff13c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:40.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.731+0000 7ff14f7fe640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 0x7ff13c06df20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff144000f60 tx=0x7ff144007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:40.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.732+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff130005290 con 0x7ff1500cfd20 2024-04-01T05:48:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:40.742+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff140034030 con 0x7ff1500cfd20 2024-04-01T05:48:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.386+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff130005580 con 0x7ff1500cfd20 2024-04-01T05:48:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.387+0000 7ff137fff640 1 -- 172.21.15.64:0/3014064783 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16645 (secure 0 0 0) 0x7ff14007aa60 con 0x7ff1500cfd20 2024-04-01T05:48:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:41.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:46.560584+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T05:48:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.394+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 msgr2=0x7ff13c06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.394+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 0x7ff13c06df20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff144000f60 tx=0x7ff144007ef0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.394+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 msgr2=0x7ff150204390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.394+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098c0 tx=0x7ff140004730 comp rx=0 tx=0).stop 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 shutdown_connections 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff13c06ba60 0x7ff13c06df20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1500d0100 0x7ff1502048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1500cfd20 0x7ff150204390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 --2- 172.21.15.64:0/3014064783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1500cc150 0x7ff15015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.396+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 >> 172.21.15.64:0/3014064783 conn(0x7ff15006b0c0 msgr2=0x7ff15016a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:41.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.397+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 shutdown_connections 2024-04-01T05:48:41.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.397+0000 7ff155d02640 1 -- 172.21.15.64:0/3014064783 wait complete. 2024-04-01T05:48:41.415 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-01T05:48:41.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T05:48:41.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.813+0000 7fd4f1125640 1 Processor -- start 2024-04-01T05:48:41.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.813+0000 7fd4f1125640 1 -- start start 2024-04-01T05:48:41.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.814+0000 7fd4f1125640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 0x7fd4ec0c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.814+0000 7fd4f1125640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 0x7fd4ec0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.814+0000 7fd4f1125640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0c5620 con 0x7fd4ec06c240 2024-04-01T05:48:41.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.814+0000 7fd4f1125640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec0b14c0 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.814+0000 7fd4f1125640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec1630d0 con 0x7fd4ec163560 2024-04-01T05:48:41.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.816+0000 7fd4ea59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd4ec06c240 0x7fd4ec0b2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35482/0 (socket says 172.21.15.64:35482) 2024-04-01T05:48:41.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.816+0000 7fd4ea59c640 1 -- 172.21.15.64:0/583074134 learned_addr learned my addr 172.21.15.64:0/583074134 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:41.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.817+0000 7fd4eb59e640 1 --2- 172.21.15.64:0/583074134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 0x7fd4ec0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:41.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.817+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621281099 0 0) 0x7fd4ec0c5620 con 0x7fd4ec06c240 2024-04-01T05:48:41.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.817+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4d4003520 con 0x7fd4ec06c240 2024-04-01T05:48:41.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.818+0000 7fd4ead9d640 1 --2- 172.21.15.64:0/583074134 >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 0x7fd4ec0c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:41.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.819+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685776471 0 0) 0x7fd4d4003520 con 0x7fd4ec06c240 2024-04-01T05:48:41.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.819+0000 7fd4eb59e640 1 -- 172.21.15.64:0/583074134 >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 msgr2=0x7fd4ec0c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:41.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.819+0000 7fd4eb59e640 1 --2- 172.21.15.64:0/583074134 >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 0x7fd4ec0c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.819+0000 7fd4eb59e640 1 -- 172.21.15.64:0/583074134 >> v1:172.21.15.64:6789/0 conn(0x7fd4ec06c240 legacy=0x7fd4ec0b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:41.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.820+0000 7fd4eb59e640 1 -- 172.21.15.64:0/583074134 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec0b3270 con 0x7fd4ec163560 2024-04-01T05:48:41.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.820+0000 7fd4ead9d640 1 --2- 172.21.15.64:0/583074134 >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 0x7fd4ec0c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.822+0000 7fd4eb59e640 1 --2- 172.21.15.64:0/583074134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 0x7fd4ec0d1bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d90 tx=0x7fd4e402bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9afbfe7dc3d8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.823+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4039070 con 0x7fd4ec163560 2024-04-01T05:48:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.823+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e402c8a0 con 0x7fd4ec163560 2024-04-01T05:48:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.823+0000 7fd4e9d9b640 1 -- 172.21.15.64:0/583074134 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc001670 con 0x7fd4ec163560 2024-04-01T05:48:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.824+0000 7fd4f1125640 1 -- 172.21.15.64:0/583074134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 msgr2=0x7fd4ec0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.824+0000 7fd4f1125640 1 --2- 172.21.15.64:0/583074134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 0x7fd4ec0d1bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d90 tx=0x7fd4e402bd70 comp rx=0 tx=0).stop 2024-04-01T05:48:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.825+0000 7fd4f1125640 1 -- 172.21.15.64:0/583074134 shutdown_connections 2024-04-01T05:48:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.825+0000 7fd4f1125640 1 --2- 172.21.15.64:0/583074134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec163560 0x7fd4ec0d1bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.825+0000 7fd4f1125640 1 --2- 172.21.15.64:0/583074134 >> v2:172.21.15.100:3300/0 conn(0x7fd4ec0c4d00 0x7fd4ec0c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.825+0000 7fd4f1125640 1 -- 172.21.15.64:0/583074134 >> 172.21.15.64:0/583074134 conn(0x7fd4ec09a980 msgr2=0x7fd4ec09ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.826+0000 7fd4f1125640 1 -- 172.21.15.64:0/583074134 shutdown_connections 2024-04-01T05:48:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.826+0000 7fd4f1125640 1 -- 172.21.15.64:0/583074134 wait complete. 2024-04-01T05:48:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.828+0000 7fd4f1125640 1 Processor -- start 2024-04-01T05:48:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.829+0000 7fd4f1125640 1 -- start start 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec06c240 0x7fd4ec1f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4ec163560 0x7fd4ec0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b3550 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec162d60 con 0x7fd4ec163560 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4f1125640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec15cc00 con 0x7fd4ec06c240 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59452/0 (socket says 172.21.15.64:59452) 2024-04-01T05:48:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.830+0000 7fd4ea59c640 1 -- 172.21.15.64:0/3426144219 learned_addr learned my addr 172.21.15.64:0/3426144219 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.831+0000 7fd4ea59c640 1 -- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec06c240 msgr2=0x7fd4ec1f9140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-01T05:48:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.831+0000 7fd4ea59c640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec06c240 0x7fd4ec1f9140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.831+0000 7fd4ea59c640 1 -- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4ec163560 msgr2=0x7fd4ec0afc80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-01T05:48:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.831+0000 7fd4ea59c640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4ec163560 0x7fd4ec0afc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.832+0000 7fd4ea59c640 1 -- 172.21.15.64:0/3426144219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec0c8340 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.832+0000 7fd4ea59c640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c700 tx=0x7fd4dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.833+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005ad0 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.833+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1f8660 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.833+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1f8c20 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.833+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc005c70 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.833+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0072b0 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.837+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd4dc007450 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.837+0000 7fd4d37fe640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 0x7fd4d806df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:41.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.837+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd4dc08a5f0 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.838+0000 7fd4ead9d640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 0x7fd4d806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:41.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.841+0000 7fd4ead9d640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 0x7fd4d806df50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009950 tx=0x7fd4e0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:41.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.853+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc005290 con 0x7fd4ec0c4d00 2024-04-01T05:48:41.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:41.862+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd4dc005e20 con 0x7fd4ec0c4d00 2024-04-01T05:48:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.746+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4cc005580 con 0x7fd4ec0c4d00 2024-04-01T05:48:42.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.747+0000 7fd4d37fe640 1 -- 172.21.15.64:0/3426144219 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7fd4dc019300 con 0x7fd4ec0c4d00 2024-04-01T05:48:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:42.749 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T05:48:42.749 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T05:48:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 msgr2=0x7fd4d806df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 0x7fd4d806df50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009950 tx=0x7fd4e0009190 comp rx=0 tx=0).stop 2024-04-01T05:48:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 msgr2=0x7fd4ec0c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c700 tx=0x7fd4dc00cbc0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 shutdown_connections 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd4d806ba90 0x7fd4d806df50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4ec163560 0x7fd4ec0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4ec0c4d00 0x7fd4ec0c7a40 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 --2- 172.21.15.64:0/3426144219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4ec06c240 0x7fd4ec1f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 >> 172.21.15.64:0/3426144219 conn(0x7fd4ec09a980 msgr2=0x7fd4ec0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 shutdown_connections 2024-04-01T05:48:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.753+0000 7fd4f1125640 1 -- 172.21.15.64:0/3426144219 wait complete. 2024-04-01T05:48:42.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T05:48:42.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.964+0000 7f99459a0640 1 Processor -- start 2024-04-01T05:48:42.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.964+0000 7f99459a0640 1 -- start start 2024-04-01T05:48:42.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f99459a0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f99400cc9f0 0x7f9940164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:42.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f99459a0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994006cd00 con 0x7f99400c6400 2024-04-01T05:48:42.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f99459a0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994006cfe0 con 0x7f99400cdcb0 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f99459a0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f99400c67e0 con 0x7f99400cc9f0 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f994499e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f99400c6400 0x7f99400cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35490/0 (socket says 172.21.15.64:35490) 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f994499e640 1 -- 172.21.15.64:0/2837776220 learned_addr learned my addr 172.21.15.64:0/2837776220 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.968+0000 7f994519f640 1 --2- 172.21.15.64:0/2837776220 >> v2:172.21.15.100:3300/0 conn(0x7f99400cc9f0 0x7f9940164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.969+0000 7f994519f640 1 -- 172.21.15.64:0/2837776220 >> v1:172.21.15.100:6789/0 conn(0x7f99400cdcb0 legacy=0x7f99400cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.969+0000 7f993f7fe640 1 -- 172.21.15.64:0/2837776220 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392676810 0 0) 0x7f994006cd00 con 0x7f99400c6400 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.969+0000 7f994519f640 1 -- 172.21.15.64:0/2837776220 >> v1:172.21.15.64:6789/0 conn(0x7f99400c6400 legacy=0x7f99400cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.969+0000 7f994519f640 1 -- 172.21.15.64:0/2837776220 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994006d2c0 con 0x7f99400cc9f0 2024-04-01T05:48:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.972+0000 7f994519f640 1 --2- 172.21.15.64:0/2837776220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cc9f0 0x7f9940164d90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9938009d90 tx=0x7f993802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e208ae8247d2b191 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.972+0000 7f993f7fe640 1 -- 172.21.15.64:0/2837776220 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938039070 con 0x7f99400cc9f0 2024-04-01T05:48:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.972+0000 7f993f7fe640 1 -- 172.21.15.64:0/2837776220 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993802cc60 con 0x7f99400cc9f0 2024-04-01T05:48:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.972+0000 7f993f7fe640 1 -- 172.21.15.64:0/2837776220 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938034660 con 0x7f99400cc9f0 2024-04-01T05:48:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.975+0000 7f99459a0640 1 -- 172.21.15.64:0/2837776220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cc9f0 msgr2=0x7f9940164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.975+0000 7f99459a0640 1 --2- 172.21.15.64:0/2837776220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cc9f0 0x7f9940164d90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9938009d90 tx=0x7f993802beb0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.975+0000 7f99459a0640 1 -- 172.21.15.64:0/2837776220 shutdown_connections 2024-04-01T05:48:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.976+0000 7f99459a0640 1 --2- 172.21.15.64:0/2837776220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cc9f0 0x7f9940164d90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.976+0000 7f99459a0640 1 -- 172.21.15.64:0/2837776220 >> 172.21.15.64:0/2837776220 conn(0x7f99400b1500 msgr2=0x7f99400b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.976+0000 7f99459a0640 1 -- 172.21.15.64:0/2837776220 shutdown_connections 2024-04-01T05:48:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.976+0000 7f99459a0640 1 -- 172.21.15.64:0/2837776220 wait complete. 2024-04-01T05:48:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.981+0000 7f99459a0640 1 Processor -- start 2024-04-01T05:48:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.981+0000 7f99459a0640 1 -- start start 2024-04-01T05:48:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.985+0000 7f99459a0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99400c6400 0x7f994015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.986+0000 7f99459a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.987+0000 7f993ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.987+0000 7f993ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59468/0 (socket says 172.21.15.64:59468) 2024-04-01T05:48:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.987+0000 7f993ffff640 1 -- 172.21.15.64:0/3118208500 learned_addr learned my addr 172.21.15.64:0/3118208500 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:42.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.987+0000 7f99459a0640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cdcb0 0x7f99400b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.988+0000 7f99459a0640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99400b1ff0 con 0x7f99400cc9f0 2024-04-01T05:48:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.988+0000 7f994519f640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cdcb0 0x7f99400b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.988+0000 7f99459a0640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f994006cab0 con 0x7f99400cdcb0 2024-04-01T05:48:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.988+0000 7f99459a0640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f99400b2170 con 0x7f99400c6400 2024-04-01T05:48:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.990+0000 7f993ffff640 1 -- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99400c6400 msgr2=0x7f994015aaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-01T05:48:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.991+0000 7f993ffff640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99400c6400 0x7f994015aaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.991+0000 7f993ffff640 1 -- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cdcb0 msgr2=0x7f99400b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.991+0000 7f993ffff640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cdcb0 0x7f99400b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.991+0000 7f993ffff640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9940154d90 con 0x7f99400cc9f0 2024-04-01T05:48:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.992+0000 7f993ffff640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f993400c7b0 tx=0x7f993400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:42.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.993+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934016020 con 0x7f99400cc9f0 2024-04-01T05:48:42.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.993+0000 7f99459a0640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9940155030 con 0x7f99400cc9f0 2024-04-01T05:48:42.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.994+0000 7f99459a0640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401558d0 con 0x7f99400cc9f0 2024-04-01T05:48:42.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.994+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934005220 con 0x7f99400cc9f0 2024-04-01T05:48:43.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.999+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934016020 con 0x7f99400cc9f0 2024-04-01T05:48:43.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:42.999+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994004f4b0 con 0x7f99400cc9f0 2024-04-01T05:48:43.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.000+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f9934007030 con 0x7f99400cc9f0 2024-04-01T05:48:43.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.002+0000 7f993d7fa640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 0x7f992406e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.003+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9934087d20 con 0x7f99400cc9f0 2024-04-01T05:48:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.003+0000 7f994499e640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 0x7f992406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.004+0000 7f994499e640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 0x7f992406e010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f992c007fd0 tx=0x7f992c007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.025+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f9934058270 con 0x7f99400cc9f0 2024-04-01T05:48:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.433+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f994006e3b0 con 0x7f99400cc9f0 2024-04-01T05:48:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.435+0000 7f993d7fa640 1 -- 172.21.15.64:0/3118208500 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7f9934058270 con 0x7f99400cc9f0 2024-04-01T05:48:43.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:43.435 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-01T00:14:44.927110+0000","modified":"2024-04-01T00:14:46.968725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5224":{"gid":5224,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1975239378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1975239378},{"type":"v1","addr":"172.21.15.100:6837","nonce":1975239378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5224,"qdb_cluster":[5224]},"id":2}]} 2024-04-01T05:48:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.438+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 msgr2=0x7f992406e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.438+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 0x7f992406e010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f992c007fd0 tx=0x7f992c007960 comp rx=0 tx=0).stop 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.438+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 msgr2=0x7f9940154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.438+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f993400c7b0 tx=0x7f993400cc70 comp rx=0 tx=0).stop 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.438+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 shutdown_connections 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.439+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f992406bb50 0x7f992406e010 secure :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f992c007fd0 tx=0x7f992c007960 comp rx=0 tx=0).stop 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.439+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99400cdcb0 0x7f99400b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.439+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99400cc9f0 0x7f9940154850 secure :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f993400c7b0 tx=0x7f993400cc70 comp rx=0 tx=0).stop 2024-04-01T05:48:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.439+0000 7f992affd640 1 --2- 172.21.15.64:0/3118208500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99400c6400 0x7f994015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.439+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 >> 172.21.15.64:0/3118208500 conn(0x7f99400b1500 msgr2=0x7f99400b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:43.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.442+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 shutdown_connections 2024-04-01T05:48:43.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.442+0000 7f992affd640 1 -- 172.21.15.64:0/3118208500 wait complete. 2024-04-01T05:48:43.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T05:48:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.649+0000 7f58d359e640 1 Processor -- start 2024-04-01T05:48:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.649+0000 7f58d359e640 1 -- start start 2024-04-01T05:48:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.651+0000 7f58d359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58d40d01e0 0x7f58d415ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.651+0000 7f58d359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 0x7f58d4070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.651+0000 7f58d359e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d40b4a10 con 0x7f58d4154d50 2024-04-01T05:48:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.651+0000 7f58d359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f58d40b2b10 con 0x7f58d40b4380 2024-04-01T05:48:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.651+0000 7f58d359e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f58d40b4cf0 con 0x7f58d40d01e0 2024-04-01T05:48:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.652+0000 7f58d2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 0x7f58d4070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.652+0000 7f58d2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 0x7f58d4070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40270/0 (socket says 172.21.15.64:40270) 2024-04-01T05:48:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.652+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3612034484 learned_addr learned my addr 172.21.15.64:0/3612034484 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.652+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3612034484 >> v2:172.21.15.100:3300/0 conn(0x7f58d40d01e0 0x7f58d415ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211461403 0 0) 0x7f58d40b4a10 con 0x7f58d4154d50 2024-04-01T05:48:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58bc003520 con 0x7f58d4154d50 2024-04-01T05:48:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2256557499 0 0) 0x7f58bc003520 con 0x7f58d4154d50 2024-04-01T05:48:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 >> v2:172.21.15.100:3300/0 conn(0x7f58d40d01e0 msgr2=0x7f58d415ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 --2- 172.21.15.64:0/3612034484 >> v2:172.21.15.100:3300/0 conn(0x7f58d40d01e0 0x7f58d415ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 msgr2=0x7f58d4070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 --2- 172.21.15.64:0/3612034484 >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 0x7f58d4070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d415e4b0 con 0x7f58d4154d50 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 815160443 0 0) 0x7f58c8001670 con 0x7f58d4154d50 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.654+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 >> v1:172.21.15.100:6789/0 conn(0x7f58d4154d50 legacy=0x7f58d40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc003a20 0x7f58bc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 0x7f58bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc0051c0 0x7f58bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40b4cf0 con 0x7f58bc0051c0 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc00bff0 con 0x7f58bc003a20 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc00c1f0 con 0x7f58bc004800 2024-04-01T05:48:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.655+0000 7f58d259c640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc003a20 0x7f58bc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.656+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3612034484 >> v2:172.21.15.64:3300/0 conn(0x7f58d40b4380 0x7f58d4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:48:43.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.656+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc0051c0 0x7f58bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.656+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 0x7f58bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d1d9b640 1 -- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc003a20 msgr2=0x7f58bc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc003a20 0x7f58bc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d1d9b640 1 -- 172.21.15.64:0/3612034484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc0051c0 msgr2=0x7f58bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc0051c0 0x7f58bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d1d9b640 1 -- 172.21.15.64:0/3612034484 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d415e4b0 con 0x7f58bc004800 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc0051c0 0x7f58bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:48:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.657+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3612034484 reap_dead start 2024-04-01T05:48:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.658+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 0x7f58bc004c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009a90 tx=0x7f58bc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c2c7f20ea1657b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.658+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4039070 con 0x7f58bc004800 2024-04-01T05:48:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.658+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c402c5d0 con 0x7f58bc004800 2024-04-01T05:48:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.659+0000 7f58d159a640 1 -- 172.21.15.64:0/3612034484 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4039070 con 0x7f58bc004800 2024-04-01T05:48:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.659+0000 7f58d359e640 1 -- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 msgr2=0x7f58bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.659+0000 7f58d359e640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 0x7f58bc004c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009a90 tx=0x7f58bc005830 comp rx=0 tx=0).stop 2024-04-01T05:48:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.660+0000 7f58d359e640 1 -- 172.21.15.64:0/3612034484 shutdown_connections 2024-04-01T05:48:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.660+0000 7f58d359e640 1 --2- 172.21.15.64:0/3612034484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58bc004800 0x7f58bc004c80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.660+0000 7f58d359e640 1 -- 172.21.15.64:0/3612034484 >> 172.21.15.64:0/3612034484 conn(0x7f58d406b0c0 msgr2=0x7f58d406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.662+0000 7f58d359e640 1 -- 172.21.15.64:0/3612034484 shutdown_connections 2024-04-01T05:48:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.662+0000 7f58d359e640 1 -- 172.21.15.64:0/3612034484 wait complete. 2024-04-01T05:48:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.664+0000 7f58d359e640 1 Processor -- start 2024-04-01T05:48:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.664+0000 7f58d359e640 1 -- start start 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.665+0000 7f58d359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc004800 0x7f58d41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.665+0000 7f58d359e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40cd740 0x7f58d414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58d415ea70 con 0x7f58d4157d80 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d359e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40b47b0 con 0x7f58bc004800 2024-04-01T05:48:43.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d359e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f58d4155130 con 0x7f58d40cd740 2024-04-01T05:48:43.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40286/0 (socket says 172.21.15.64:40286) 2024-04-01T05:48:43.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3186685518 learned_addr learned my addr 172.21.15.64:0/3186685518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:43.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.666+0000 7f58d1d9b640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40cd740 0x7f58d414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40cd740 msgr2=0x7f58d414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40cd740 0x7f58d414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc004800 msgr2=0x7f58d41560b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc004800 0x7f58d41560b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 -- 172.21.15.64:0/3186685518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d4156750 con 0x7f58d4157d80 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.667+0000 7f58d2d9d640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f58d40d06d0 tx=0x7f58cc007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.668+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01a070 con 0x7f58d4157d80 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.668+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc003da0 con 0x7f58d4157d80 2024-04-01T05:48:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.668+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc004270 con 0x7f58d4157d80 2024-04-01T05:48:43.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.668+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d4156a30 con 0x7f58d4157d80 2024-04-01T05:48:43.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.668+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d4156ff0 con 0x7f58d4157d80 2024-04-01T05:48:43.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.670+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f58cc01d070 con 0x7f58d4157d80 2024-04-01T05:48:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.671+0000 7f58c2ffd640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 0x7f58bc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.671+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f58cc089da0 con 0x7f58d4157d80 2024-04-01T05:48:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.672+0000 7f58d259c640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 0x7f58bc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.672+0000 7f58d259c640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 0x7f58bc06f9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000fb0 tx=0x7f58c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.673+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b8005290 con 0x7f58d4157d80 2024-04-01T05:48:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:43.680+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f58cc012070 con 0x7f58d4157d80 2024-04-01T05:48:44.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.647+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58b8005580 con 0x7f58d4157d80 2024-04-01T05:48:44.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.652+0000 7f58c2ffd640 1 -- 172.21.15.64:0/3186685518 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16645 (secure 0 0 0) 0x7f58cc0583d0 con 0x7f58d4157d80 2024-04-01T05:48:44.653 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:44.653 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T00:14:46.560584+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-01T00:14:43.503546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-01T00:14:43.889938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T05:48:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.659+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 msgr2=0x7f58bc06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.659+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 0x7f58bc06f9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000fb0 tx=0x7f58c8009040 comp rx=0 tx=0).stop 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.659+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 msgr2=0x7f58d4158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.659+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f58d40d06d0 tx=0x7f58cc007e20 comp rx=0 tx=0).stop 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 shutdown_connections 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f58bc06d500 0x7f58bc06f9c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d4157d80 0x7f58d4158200 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40cd740 0x7f58d414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 --2- 172.21.15.64:0/3186685518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58bc004800 0x7f58d41560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.660+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 >> 172.21.15.64:0/3186685518 conn(0x7f58d406b0c0 msgr2=0x7f58d4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:44.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.661+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 shutdown_connections 2024-04-01T05:48:44.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.666+0000 7f58d359e640 1 -- 172.21.15.64:0/3186685518 wait complete. 2024-04-01T05:48:44.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-01T05:48:44.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.936+0000 7f462d0d2640 1 Processor -- start 2024-04-01T05:48:44.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.937+0000 7f462d0d2640 1 -- start start 2024-04-01T05:48:44.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 0x7f462815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f462802bff0 0x7f46280d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802c460 0x7f46280d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f462806daf0 con 0x7f462815ea90 2024-04-01T05:48:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f462806dd90 con 0x7f462802bff0 2024-04-01T05:48:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462d0d2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f462806e070 con 0x7f462802c460 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f4626d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 0x7f462815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f4626d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 0x7f462815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40300/0 (socket says 172.21.15.64:40300) 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f4626d9d640 1 -- 172.21.15.64:0/3091311195 learned_addr learned my addr 172.21.15.64:0/3091311195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462659c640 1 --2- 172.21.15.64:0/3091311195 >> v2:172.21.15.100:3300/0 conn(0x7f462802bff0 0x7f46280d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.938+0000 7f462759e640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802c460 0x7f46280d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.939+0000 7f462659c640 1 -- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802c460 msgr2=0x7f46280d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.939+0000 7f462659c640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802c460 0x7f46280d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.939+0000 7f462659c640 1 -- 172.21.15.64:0/3091311195 >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 msgr2=0x7f462815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.939+0000 7f462659c640 1 --2- 172.21.15.64:0/3091311195 >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 0x7f462815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.939+0000 7f462659c640 1 -- 172.21.15.64:0/3091311195 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462806e350 con 0x7f462802bff0 2024-04-01T05:48:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.940+0000 7f462659c640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462802bff0 0x7f46280d0060 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f46280b5940 tx=0x7f461002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4504d9296e884fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.940+0000 7f4625d9b640 1 -- 172.21.15.64:0/3091311195 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100315a0 con 0x7f462802bff0 2024-04-01T05:48:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.941+0000 7f4625d9b640 1 -- 172.21.15.64:0/3091311195 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4610037070 con 0x7f462802bff0 2024-04-01T05:48:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.941+0000 7f4625d9b640 1 -- 172.21.15.64:0/3091311195 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100315a0 con 0x7f462802bff0 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 -- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462802bff0 msgr2=0x7f46280d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462802bff0 0x7f46280d0060 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f46280b5940 tx=0x7f461002b840 comp rx=0 tx=0).stop 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 -- 172.21.15.64:0/3091311195 shutdown_connections 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802c460 0x7f46280d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3091311195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462802bff0 0x7f46280d0060 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3091311195 >> v2:172.21.15.64:3300/0 conn(0x7f462815ea90 0x7f462815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.942+0000 7f462d0d2640 1 -- 172.21.15.64:0/3091311195 >> 172.21.15.64:0/3091311195 conn(0x7f462802ceb0 msgr2=0x7f462806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.943+0000 7f462d0d2640 1 -- 172.21.15.64:0/3091311195 shutdown_connections 2024-04-01T05:48:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.943+0000 7f462d0d2640 1 -- 172.21.15.64:0/3091311195 wait complete. 2024-04-01T05:48:44.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.949+0000 7f462d0d2640 1 Processor -- start 2024-04-01T05:48:44.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.950+0000 7f462d0d2640 1 -- start start 2024-04-01T05:48:44.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.954+0000 7f462d0d2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 0x7f46280b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.955+0000 7f462d0d2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462802c460 0x7f46280cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.955+0000 7f4626d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 0x7f46280b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.955+0000 7f4626d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 0x7f46280b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:40100/0 (socket says 172.21.15.64:40100) 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.955+0000 7f4626d9d640 1 -- 172.21.15.64:0/3461938808 learned_addr learned my addr 172.21.15.64:0/3461938808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.955+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 0x7f462814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.956+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f462815a890 con 0x7f462802c460 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.956+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f462806e910 con 0x7f462815ea90 2024-04-01T05:48:44.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.956+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f462815ee70 con 0x7f462802bff0 2024-04-01T05:48:44.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.956+0000 7f462659c640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462802c460 0x7f46280cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.956+0000 7f462759e640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 0x7f462814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.957+0000 7f462759e640 1 -- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 msgr2=0x7f46280b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.957+0000 7f462759e640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 0x7f46280b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.957+0000 7f462759e640 1 -- 172.21.15.64:0/3461938808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462802c460 msgr2=0x7f46280cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.957+0000 7f462759e640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462802c460 0x7f46280cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.957+0000 7f462759e640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462815acb0 con 0x7f462815ea90 2024-04-01T05:48:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.958+0000 7f462759e640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 0x7f462814bab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4620007d10 tx=0x7f462000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.960+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620019070 con 0x7f462815ea90 2024-04-01T05:48:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.960+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4620003990 con 0x7f462815ea90 2024-04-01T05:48:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.960+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003d80 con 0x7f462815ea90 2024-04-01T05:48:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.960+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46280cea30 con 0x7f462815ea90 2024-04-01T05:48:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.960+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46280ceff0 con 0x7f462815ea90 2024-04-01T05:48:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.964+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46280688b0 con 0x7f462815ea90 2024-04-01T05:48:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.966+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f462001c070 con 0x7f462815ea90 2024-04-01T05:48:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.967+0000 7f461f7fe640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 0x7f460806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.967+0000 7f4626d9d640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 0x7f460806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.968+0000 7f4626d9d640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 0x7f460806de50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4618006f40 tx=0x7f4618009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.968+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f462008a4f0 con 0x7f462815ea90 2024-04-01T05:48:44.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:44.974+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f4620058aa0 con 0x7f462815ea90 2024-04-01T05:48:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.407+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46280afde0 con 0x7f462815ea90 2024-04-01T05:48:45.409 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-04-01T05:48:45.409+0000 7f75af780640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-01T05:48:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.422+0000 7f461f7fe640 1 -- 172.21.15.64:0/3461938808 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7f462005cb40 con 0x7f462815ea90 2024-04-01T05:48:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-01T05:48:45.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 msgr2=0x7f460806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 0x7f460806de50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4618006f40 tx=0x7f4618009240 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 msgr2=0x7f462814bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 0x7f462814bab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4620007d10 tx=0x7f462000e4b0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 shutdown_connections 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f460806b990 0x7f460806de50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f462815ea90 0x7f462814bab0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462802c460 0x7f46280cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 --2- 172.21.15.64:0/3461938808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f462802bff0 0x7f46280b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.428+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 >> 172.21.15.64:0/3461938808 conn(0x7f462802ceb0 msgr2=0x7f4628157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.432+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 shutdown_connections 2024-04-01T05:48:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.433+0000 7f462d0d2640 1 -- 172.21.15.64:0/3461938808 wait complete. 2024-04-01T05:48:45.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-01T05:48:45.650 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-01T05:48:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.728+0000 7f422417a640 1 Processor -- start 2024-04-01T05:48:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.728+0000 7f422417a640 1 -- start start 2024-04-01T05:48:45.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.730+0000 7f422417a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 0x7f421c0cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.730+0000 7f422417a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f421c0cc2b0 0x7f421c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.730+0000 7f422417a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421c0cee20 con 0x7f421c0af5b0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.730+0000 7f422417a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f421c02c700 con 0x7f421c0cc2b0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.730+0000 7f422417a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0cf550 con 0x7f421c0af990 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221f16640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f421c0af5b0 0x7f421c02bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:49880/0 (socket says 172.21.15.64:49880) 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221715640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 0x7f421c0cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221f16640 1 -- 172.21.15.64:0/1586737501 learned_addr learned my addr 172.21.15.64:0/1586737501 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4222717640 1 --2- 172.21.15.64:0/1586737501 >> v2:172.21.15.100:3300/0 conn(0x7f421c0cc2b0 0x7f421c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4220f14640 1 -- 172.21.15.64:0/1586737501 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292612896 0 0) 0x7f421c0cee20 con 0x7f421c0af5b0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4220f14640 1 -- 172.21.15.64:0/1586737501 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4208003520 con 0x7f421c0af5b0 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221715640 1 -- 172.21.15.64:0/1586737501 >> v2:172.21.15.100:3300/0 conn(0x7f421c0cc2b0 msgr2=0x7f421c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221715640 1 --2- 172.21.15.64:0/1586737501 >> v2:172.21.15.100:3300/0 conn(0x7f421c0cc2b0 0x7f421c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221715640 1 -- 172.21.15.64:0/1586737501 >> v1:172.21.15.100:6789/0 conn(0x7f421c0af5b0 legacy=0x7f421c02bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.731+0000 7f4221715640 1 -- 172.21.15.64:0/1586737501 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c0cf7a0 con 0x7f421c0af990 2024-04-01T05:48:45.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.733+0000 7f4221715640 1 --2- 172.21.15.64:0/1586737501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 0x7f421c0cbd70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097a0 tx=0x7f420c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599fc71e2b158cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:45.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.733+0000 7f4220f14640 1 -- 172.21.15.64:0/1586737501 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0315a0 con 0x7f421c0af990 2024-04-01T05:48:45.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.733+0000 7f4220f14640 1 -- 172.21.15.64:0/1586737501 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c037070 con 0x7f421c0af990 2024-04-01T05:48:45.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.733+0000 7f4220f14640 1 -- 172.21.15.64:0/1586737501 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c031960 con 0x7f421c0af990 2024-04-01T05:48:45.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 -- 172.21.15.64:0/1586737501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 msgr2=0x7f421c0cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 --2- 172.21.15.64:0/1586737501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 0x7f421c0cbd70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097a0 tx=0x7f420c02b840 comp rx=0 tx=0).stop 2024-04-01T05:48:45.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 -- 172.21.15.64:0/1586737501 shutdown_connections 2024-04-01T05:48:45.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 --2- 172.21.15.64:0/1586737501 >> v2:172.21.15.100:3300/0 conn(0x7f421c0cc2b0 0x7f421c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 --2- 172.21.15.64:0/1586737501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0af990 0x7f421c0cbd70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 -- 172.21.15.64:0/1586737501 >> 172.21.15.64:0/1586737501 conn(0x7f421c072890 msgr2=0x7f421c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 -- 172.21.15.64:0/1586737501 shutdown_connections 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.734+0000 7f422417a640 1 -- 172.21.15.64:0/1586737501 wait complete. 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.735+0000 7f422417a640 1 Processor -- start 2024-04-01T05:48:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.735+0000 7f422417a640 1 -- start start 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 0x7f421c0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 0x7f421c099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 0x7f421c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0cea30 con 0x7f421c0af5b0 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0ce8b0 con 0x7f421c0af990 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f422417a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0cebb0 con 0x7f421c0c3910 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f4221f16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 0x7f421c0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f4221715640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 0x7f421c099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f4221f16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 0x7f421c0afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40314/0 (socket says 172.21.15.64:40314) 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f4221f16640 1 -- 172.21.15.64:0/1805012893 learned_addr learned my addr 172.21.15.64:0/1805012893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:45.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.736+0000 7f4222717640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 0x7f421c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 -- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 msgr2=0x7f421c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 0x7f421c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 -- 172.21.15.64:0/1805012893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 msgr2=0x7f421c0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 0x7f421c0afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 -- 172.21.15.64:0/1805012893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c0b02b0 con 0x7f421c0af990 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4222717640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 0x7f421c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:48:45.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f4221715640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 0x7f421c099860 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f421c0cf6d0 tx=0x7f420c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c032630 con 0x7f421c0af990 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0327f0 con 0x7f421c0af990 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c0c96d0 con 0x7f421c0af990 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c032be0 con 0x7f421c0af990 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.737+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c0b3ad0 con 0x7f421c0af990 2024-04-01T05:48:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.739+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c0b3900 con 0x7f421c0af990 2024-04-01T05:48:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.740+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f420c03b3f0 con 0x7f421c0af990 2024-04-01T05:48:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.741+0000 7f42127fc640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 0x7f420006de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.741+0000 7f4221f16640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 0x7f420006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:45.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.741+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f420c02f030 con 0x7f421c0af990 2024-04-01T05:48:45.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.742+0000 7f4221f16640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 0x7f420006de80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4218001560 tx=0x7f4218001b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:45.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:45.745+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f420c040070 con 0x7f421c0af990 2024-04-01T05:48:46.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.398+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f421c0c9bf0 con 0x7f421c0af990 2024-04-01T05:48:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.422+0000 7f42127fc640 1 -- 172.21.15.64:0/1805012893 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f420c07ae50 con 0x7f421c0af990 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 msgr2=0x7f420006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 0x7f420006de80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4218001560 tx=0x7f4218001b50 comp rx=0 tx=0).stop 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 msgr2=0x7f421c099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 0x7f421c099860 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f421c0cf6d0 tx=0x7f420c0046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 shutdown_connections 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f420006b9c0 0x7f420006de80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f421c0c3910 0x7f421c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c0af990 0x7f421c099860 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.425+0000 7f422417a640 1 --2- 172.21.15.64:0/1805012893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421c0af5b0 0x7f421c0afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.426+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 >> 172.21.15.64:0/1805012893 conn(0x7f421c072890 msgr2=0x7f421c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.426+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 shutdown_connections 2024-04-01T05:48:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.426+0000 7f422417a640 1 -- 172.21.15.64:0/1805012893 wait complete. 2024-04-01T05:48:46.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-01T05:48:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.716+0000 7f8471b81640 1 Processor -- start 2024-04-01T05:48:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.717+0000 7f8471b81640 1 -- start start 2024-04-01T05:48:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f8471b81640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f846c15f1e0 0x7f846c0ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f8471b81640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c0adf70 con 0x7f846c06dd70 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f8471b81640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c0ae250 con 0x7f846c158660 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f8471b81640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f846c15f650 con 0x7f846c15f1e0 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f846affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f846c06dd70 0x7f846c071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39200/0 (socket says 172.21.15.64:39200) 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f846affd640 1 -- 172.21.15.64:0/1610236952 learned_addr learned my addr 172.21.15.64:0/1610236952 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.719+0000 7f846bfff640 1 --2- 172.21.15.64:0/1610236952 >> v2:172.21.15.100:3300/0 conn(0x7f846c15f1e0 0x7f846c0ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.720+0000 7f846bfff640 1 -- 172.21.15.64:0/1610236952 >> v1:172.21.15.100:6789/0 conn(0x7f846c158660 legacy=0x7f846c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.720+0000 7f846bfff640 1 -- 172.21.15.64:0/1610236952 >> v1:172.21.15.64:6789/0 conn(0x7f846c06dd70 legacy=0x7f846c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.720+0000 7f846bfff640 1 -- 172.21.15.64:0/1610236952 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c0ae530 con 0x7f846c15f1e0 2024-04-01T05:48:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.720+0000 7f846a7fc640 1 -- 172.21.15.64:0/1610236952 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461640601 0 0) 0x7f846c0ae250 con 0x7f846c158660 2024-04-01T05:48:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.721+0000 7f846bfff640 1 --2- 172.21.15.64:0/1610236952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0ad7f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f846402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b3bb59d2d49693 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.721+0000 7f846a7fc640 1 -- 172.21.15.64:0/1610236952 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464039070 con 0x7f846c15f1e0 2024-04-01T05:48:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.721+0000 7f846a7fc640 1 -- 172.21.15.64:0/1610236952 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846402ccf0 con 0x7f846c15f1e0 2024-04-01T05:48:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.721+0000 7f846a7fc640 1 -- 172.21.15.64:0/1610236952 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464034660 con 0x7f846c15f1e0 2024-04-01T05:48:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 -- 172.21.15.64:0/1610236952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 msgr2=0x7f846c0ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 --2- 172.21.15.64:0/1610236952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0ad7f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f846402bf40 comp rx=0 tx=0).stop 2024-04-01T05:48:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 -- 172.21.15.64:0/1610236952 shutdown_connections 2024-04-01T05:48:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 --2- 172.21.15.64:0/1610236952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0ad7f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 -- 172.21.15.64:0/1610236952 >> 172.21.15.64:0/1610236952 conn(0x7f846c0b1730 msgr2=0x7f846c0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 -- 172.21.15.64:0/1610236952 shutdown_connections 2024-04-01T05:48:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.723+0000 7f8471b81640 1 -- 172.21.15.64:0/1610236952 wait complete. 2024-04-01T05:48:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.726+0000 7f8471b81640 1 Processor -- start 2024-04-01T05:48:46.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.727+0000 7f8471b81640 1 -- start start 2024-04-01T05:48:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 0x7f846c1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f846c158660 0x7f846c162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f846c0aeb80 con 0x7f846c06dd70 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f846c0add30 con 0x7f846c15f1e0 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f8471b81640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f846c158a40 con 0x7f846c158660 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 0x7f846c1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846affd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f846c158660 0x7f846c162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:58272/0 (socket says 172.21.15.64:58272) 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846bfff640 1 -- 172.21.15.64:0/3563732560 learned_addr learned my addr 172.21.15.64:0/3563732560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.729+0000 7f846b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 0x7f846c1f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40324/0 (socket says 172.21.15.64:40324) 2024-04-01T05:48:46.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 -- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f846c158660 msgr2=0x7f846c162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f846c158660 0x7f846c162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 -- 172.21.15.64:0/3563732560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 msgr2=0x7f846c1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 0x7f846c1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 -- 172.21.15.64:0/3563732560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c153250 con 0x7f846c15f1e0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f846bfff640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f8464004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464039050 con 0x7f846c15f1e0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846403a040 con 0x7f846c15f1e0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846403e610 con 0x7f846c15f1e0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.730+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c151e50 con 0x7f846c15f1e0 2024-04-01T05:48:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.731+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1627c0 con 0x7f846c15f1e0 2024-04-01T05:48:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.732+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f8464047020 con 0x7f846c15f1e0 2024-04-01T05:48:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.733+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c151c40 con 0x7f846c15f1e0 2024-04-01T05:48:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.733+0000 7f8453fff640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 0x7f845806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.734+0000 7f846b7fe640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 0x7f845806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.734+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f84640acc30 con 0x7f846c15f1e0 2024-04-01T05:48:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.735+0000 7f846b7fe640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 0x7f845806de50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f845c000f60 tx=0x7f845c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:46.739+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f846404e020 con 0x7f846c15f1e0 2024-04-01T05:48:47.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:47.280+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f846c0681a0 con 0x7f846c15f1e0 2024-04-01T05:48:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.067+0000 7f8453fff640 1 -- 172.21.15.64:0/3563732560 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f846407b1b0 con 0x7f846c15f1e0 2024-04-01T05:48:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-04-01T05:48:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.075+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 msgr2=0x7f845806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.075+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 0x7f845806de50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f845c000f60 tx=0x7f845c009040 comp rx=0 tx=0).stop 2024-04-01T05:48:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 msgr2=0x7f846c0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f8464004a40 comp rx=0 tx=0).stop 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 shutdown_connections 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f845806b990 0x7f845806de50 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f845c000f60 tx=0x7f845c009040 comp rx=0 tx=0).stop 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f846c15f1e0 0x7f846c0b3fa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f846c158660 0x7f846c162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 --2- 172.21.15.64:0/3563732560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f846c06dd70 0x7f846c1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.076+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 >> 172.21.15.64:0/3563732560 conn(0x7f846c0b1730 msgr2=0x7f846c15f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.077+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 shutdown_connections 2024-04-01T05:48:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.077+0000 7f8471b81640 1 -- 172.21.15.64:0/3563732560 wait complete. 2024-04-01T05:48:48.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-01T05:48:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.489+0000 7f3ec1a9a640 1 Processor -- start 2024-04-01T05:48:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.489+0000 7f3ec1a9a640 1 -- start start 2024-04-01T05:48:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.491+0000 7f3ec1a9a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 0x7f3ebc0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ec1a9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ec1a9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc0d1e20 0x7f3ebc0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ec1a9a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f3ebc0d4d90 con 0x7f3ebc1618b0 2024-04-01T05:48:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ec1a9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3ebc0d1530 con 0x7f3ebc0cf070 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ec1a9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3ebc0cfa20 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ebaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ebb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 0x7f3ebc0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ebaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:58288/0 (socket says 172.21.15.64:58288) 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2832145836 learned_addr learned my addr 172.21.15.64:0/2832145836 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.492+0000 7f3ebbfff640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc0d1e20 0x7f3ebc0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc0d1e20 msgr2=0x7f3ebc0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc0d1e20 0x7f3ebc0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2832145836 >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 msgr2=0x7f3ebc0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:48.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2832145836 >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 0x7f3ebc0ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2832145836 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc0b3260 con 0x7f3ebc0cf070 2024-04-01T05:48:48.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.493+0000 7f3ebb7fe640 1 --2- 172.21.15.64:0/2832145836 >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 0x7f3ebc0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:48:48.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.495+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc02cfb0 tx=0x7f3eb002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d285805d2e2264 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:48.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.495+0000 7f3eba7fc640 1 -- 172.21.15.64:0/2832145836 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0035020 con 0x7f3ebc0cf070 2024-04-01T05:48:48.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.495+0000 7f3eba7fc640 1 -- 172.21.15.64:0/2832145836 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb0031870 con 0x7f3ebc0cf070 2024-04-01T05:48:48.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.495+0000 7f3eba7fc640 1 -- 172.21.15.64:0/2832145836 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0031c40 con 0x7f3ebc0cf070 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0cf070 msgr2=0x7f3ebc0cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc02cfb0 tx=0x7f3eb002b910 comp rx=0 tx=0).stop 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2832145836 shutdown_connections 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc0d1e20 0x7f3ebc0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2832145836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc0cf4e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2832145836 >> v2:172.21.15.64:3300/0 conn(0x7f3ebc1618b0 0x7f3ebc0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2832145836 >> 172.21.15.64:0/2832145836 conn(0x7f3ebc0d3f20 msgr2=0x7f3ebc0d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2832145836 shutdown_connections 2024-04-01T05:48:48.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.497+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2832145836 wait complete. 2024-04-01T05:48:48.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.501+0000 7f3ec1a9a640 1 Processor -- start 2024-04-01T05:48:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.502+0000 7f3ec1a9a640 1 -- start start 2024-04-01T05:48:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 0x7f3ebc0b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc0b3540 con 0x7f3ebc0cf070 2024-04-01T05:48:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc0d11c0 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.504+0000 7f3ec1a9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3ebc161c90 con 0x7f3ebc1618b0 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:58302/0 (socket says 172.21.15.64:58302) 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2997450722 learned_addr learned my addr 172.21.15.64:0/2997450722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebbfff640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 0x7f3ebc0b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 msgr2=0x7f3ebc0b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 0x7f3ebc0b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2997450722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0cf070 msgr2=0x7f3ebc152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebaffd640 1 -- 172.21.15.64:0/2997450722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc0c7b00 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.505+0000 7f3ebbfff640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 0x7f3ebc0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.506+0000 7f3ebaffd640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3eb002b950 tx=0x7f3eb00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.506+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0040070 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.506+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb0030d00 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.506+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ebc0b9db0 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.506+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0035040 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.507+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ebc0b9ad0 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.508+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ebc0688b0 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.509+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3eb004b060 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.511+0000 7f3ea3fff640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 0x7f3ea806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:48.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.511+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f3eb00ac1a0 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.511+0000 7f3ebb7fe640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 0x7f3ea806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:48.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.512+0000 7f3ebb7fe640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 0x7f3ea806de80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006e80 tx=0x7f3eac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:48.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.515+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f3eb007a950 con 0x7f3ebc0d1e20 2024-04-01T05:48:48.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:48.774+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ebc1610d0 con 0x7f3ebc0d1e20 2024-04-01T05:48:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.082+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2997450722 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f3eb007e9f0 con 0x7f3ebc0d1e20 2024-04-01T05:48:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-04-01T05:48:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 msgr2=0x7f3ea806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 0x7f3ea806de80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006e80 tx=0x7f3eac008040 comp rx=0 tx=0).stop 2024-04-01T05:48:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 msgr2=0x7f3ebc0c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3eb002b950 tx=0x7f3eb00046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 shutdown_connections 2024-04-01T05:48:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f3ea806b9c0 0x7f3ea806de80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ebc1618b0 0x7f3ebc0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ebc0d1e20 0x7f3ebc0c75c0 secure :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3eb002b950 tx=0x7f3eb00046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.089+0000 7f3ec1a9a640 1 --2- 172.21.15.64:0/2997450722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0cf070 0x7f3ebc152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.090+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 >> 172.21.15.64:0/2997450722 conn(0x7f3ebc0d3f20 msgr2=0x7f3ebc0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.090+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 shutdown_connections 2024-04-01T05:48:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.091+0000 7f3ec1a9a640 1 -- 172.21.15.64:0/2997450722 wait complete. 2024-04-01T05:48:49.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-01T05:48:49.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.569+0000 7f37b1715640 1 Processor -- start 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.573+0000 7f37b1715640 1 -- start start 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.577+0000 7f37b1715640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37ac0ca860 0x7f37ac0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.577+0000 7f37b1715640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac06f110 con 0x7f37ac159fc0 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.577+0000 7f37b1715640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac15a3a0 con 0x7f37ac071c90 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.577+0000 7f37b1715640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f37ac071260 con 0x7f37ac0ca860 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.577+0000 7f37aa7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37ac0ca860 0x7f37ac0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.578+0000 7f37aa7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37ac0ca860 0x7f37ac0cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40356/0 (socket says 172.21.15.64:40356) 2024-04-01T05:48:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.578+0000 7f37aa7fc640 1 -- 172.21.15.64:0/1343408450 learned_addr learned my addr 172.21.15.64:0/1343408450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.579+0000 7f37aa7fc640 1 -- 172.21.15.64:0/1343408450 >> v1:172.21.15.100:6789/0 conn(0x7f37ac071c90 legacy=0x7f37ac0720b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T05:48:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.579+0000 7f37aa7fc640 1 -- 172.21.15.64:0/1343408450 >> v1:172.21.15.64:6789/0 conn(0x7f37ac159fc0 legacy=0x7f37ac0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.579+0000 7f37aa7fc640 1 -- 172.21.15.64:0/1343408450 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac0c5e00 con 0x7f37ac0ca860 2024-04-01T05:48:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.584+0000 7f37aa7fc640 1 --2- 172.21.15.64:0/1343408450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac0ca860 0x7f37ac0cacc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f37ac070780 tx=0x7f37a002cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e014cfc46c5912a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.589+0000 7f37a9ffb640 1 -- 172.21.15.64:0/1343408450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a003a070 con 0x7f37ac0ca860 2024-04-01T05:48:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.589+0000 7f37a9ffb640 1 -- 172.21.15.64:0/1343408450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a002d9d0 con 0x7f37ac0ca860 2024-04-01T05:48:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.591+0000 7f37b1715640 1 -- 172.21.15.64:0/1343408450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac0ca860 msgr2=0x7f37ac0cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.591+0000 7f37b1715640 1 --2- 172.21.15.64:0/1343408450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac0ca860 0x7f37ac0cacc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f37ac070780 tx=0x7f37a002cc20 comp rx=0 tx=0).stop 2024-04-01T05:48:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.597+0000 7f37b1715640 1 -- 172.21.15.64:0/1343408450 shutdown_connections 2024-04-01T05:48:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.597+0000 7f37b1715640 1 --2- 172.21.15.64:0/1343408450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac0ca860 0x7f37ac0cacc0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.597+0000 7f37b1715640 1 -- 172.21.15.64:0/1343408450 >> 172.21.15.64:0/1343408450 conn(0x7f37ac0cfa90 msgr2=0x7f37ac0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.601+0000 7f37b1715640 1 -- 172.21.15.64:0/1343408450 shutdown_connections 2024-04-01T05:48:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.601+0000 7f37b1715640 1 -- 172.21.15.64:0/1343408450 wait complete. 2024-04-01T05:48:49.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.606+0000 7f37b1715640 1 Processor -- start 2024-04-01T05:48:49.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.607+0000 7f37b1715640 1 -- start start 2024-04-01T05:48:49.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac071c90 0x7f37ac155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 0x7f37ac146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 0x7f37ac146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac0c63c0 con 0x7f37ac071c90 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac071490 con 0x7f37ac0ca860 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.608+0000 7f37b1715640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac071770 con 0x7f37ac159fc0 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.609+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 0x7f37ac146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.609+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 0x7f37ac146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:40156/0 (socket says 172.21.15.64:40156) 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.609+0000 7f37ab7fe640 1 -- 172.21.15.64:0/264796929 learned_addr learned my addr 172.21.15.64:0/264796929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.609+0000 7f37aa7fc640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 0x7f37ac146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 -- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 msgr2=0x7f37ac146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 0x7f37ac146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 -- 172.21.15.64:0/264796929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac071c90 msgr2=0x7f37ac155b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-01T05:48:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac071c90 0x7f37ac155b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 -- 172.21.15.64:0/264796929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac156050 con 0x7f37ac0ca860 2024-04-01T05:48:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.610+0000 7f37aa7fc640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 0x7f37ac146030 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f37a0034000 tx=0x7f37a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.612+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a003a040 con 0x7f37ac0ca860 2024-04-01T05:48:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.612+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac167ef0 con 0x7f37ac0ca860 2024-04-01T05:48:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.612+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac0c7090 con 0x7f37ac0ca860 2024-04-01T05:48:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.612+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a0045070 con 0x7f37ac0ca860 2024-04-01T05:48:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.612+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0040890 con 0x7f37ac0ca860 2024-04-01T05:48:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.627+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c005290 con 0x7f37ac0ca860 2024-04-01T05:48:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.639+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f37a002d950 con 0x7f37ac0ca860 2024-04-01T05:48:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.639+0000 7f37937fe640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 0x7f379806df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.639+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f37a00ac7b0 con 0x7f37ac0ca860 2024-04-01T05:48:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.641+0000 7f37aaffd640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 0x7f379806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:49.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.644+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f37a007b730 con 0x7f37ac0ca860 2024-04-01T05:48:49.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:49.645+0000 7f37aaffd640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 0x7f379806df20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f37a400abb0 tx=0x7f37a4009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.775+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f378c005d90 con 0x7f37ac0ca860 2024-04-01T05:48:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.789+0000 7f37937fe640 1 -- 172.21.15.64:0/264796929 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f37a007b030 con 0x7f37ac0ca860 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 msgr2=0x7f379806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 0x7f379806df20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f37a400abb0 tx=0x7f37a4009250 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 msgr2=0x7f37ac146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 0x7f37ac146030 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f37a0034000 tx=0x7f37a00046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 shutdown_connections 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f379806ba60 0x7f379806df20 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f37a400abb0 tx=0x7f37a4009250 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37ac159fc0 0x7f37ac146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.795+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ac0ca860 0x7f37ac146030 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.796+0000 7f37b1715640 1 --2- 172.21.15.64:0/264796929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac071c90 0x7f37ac155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.796+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 >> 172.21.15.64:0/264796929 conn(0x7f37ac0cfa90 msgr2=0x7f37ac0727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.800+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 shutdown_connections 2024-04-01T05:48:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:50.802+0000 7f37b1715640 1 -- 172.21.15.64:0/264796929 wait complete. 2024-04-01T05:48:50.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-01T05:48:51.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.320+0000 7fb80ec4e640 1 Processor -- start 2024-04-01T05:48:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.321+0000 7fb80ec4e640 1 -- start start 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.329+0000 7fb80ec4e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb80815dcf0 0x7fb808150d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.329+0000 7fb80ec4e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8081512a0 0x7fb80815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.329+0000 7fb80ec4e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb80815e360 con 0x7fb80806bbb0 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.329+0000 7fb80ec4e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb8080cf4a0 con 0x7fb80815dcf0 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.329+0000 7fb80ec4e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb80815e640 con 0x7fb8081512a0 2024-04-01T05:48:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb80815dcf0 0x7fb808150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb80815dcf0 0x7fb808150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:58318/0 (socket says 172.21.15.64:58318) 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 -- 172.21.15.64:0/2173464651 learned_addr learned my addr 172.21.15.64:0/2173464651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 -- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8081512a0 msgr2=0x7fb80815ad30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 --2- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8081512a0 0x7fb80815ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.330+0000 7fb807fff640 1 -- 172.21.15.64:0/2173464651 >> v1:172.21.15.100:6789/0 conn(0x7fb80806bbb0 legacy=0x7fb80806ca20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-01T05:48:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.331+0000 7fb807fff640 1 -- 172.21.15.64:0/2173464651 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80815e920 con 0x7fb80815dcf0 2024-04-01T05:48:51.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.336+0000 7fb807fff640 1 --2- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb80815dcf0 0x7fb808150d60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e50 tx=0x7fb80003cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a58b6dce3505b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.339+0000 7fb8077fe640 1 -- 172.21.15.64:0/2173464651 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80003dab0 con 0x7fb80815dcf0 2024-04-01T05:48:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.339+0000 7fb8077fe640 1 -- 172.21.15.64:0/2173464651 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80003dc50 con 0x7fb80815dcf0 2024-04-01T05:48:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.339+0000 7fb8077fe640 1 -- 172.21.15.64:0/2173464651 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8000453f0 con 0x7fb80815dcf0 2024-04-01T05:48:51.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.341+0000 7fb80ec4e640 1 -- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb80815dcf0 msgr2=0x7fb808150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.341+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb80815dcf0 0x7fb808150d60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e50 tx=0x7fb80003cf10 comp rx=0 tx=0).stop 2024-04-01T05:48:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.342+0000 7fb80ec4e640 1 -- 172.21.15.64:0/2173464651 shutdown_connections 2024-04-01T05:48:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.342+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb80815dcf0 0x7fb808150d60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.342+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/2173464651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8081512a0 0x7fb80815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.342+0000 7fb80ec4e640 1 -- 172.21.15.64:0/2173464651 >> 172.21.15.64:0/2173464651 conn(0x7fb80806b0c0 msgr2=0x7fb80806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.351+0000 7fb80ec4e640 1 -- 172.21.15.64:0/2173464651 shutdown_connections 2024-04-01T05:48:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.354+0000 7fb80ec4e640 1 -- 172.21.15.64:0/2173464651 wait complete. 2024-04-01T05:48:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.359+0000 7fb80ec4e640 1 Processor -- start 2024-04-01T05:48:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.361+0000 7fb80ec4e640 1 -- start start 2024-04-01T05:48:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.372+0000 7fb80ec4e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb80806bbb0 0x7fb808150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.372+0000 7fb80ec4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.372+0000 7fb80ec4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb80815dcf0 0x7fb8081464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.372+0000 7fb80ec4e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb80806d9f0 con 0x7fb80815dcf0 2024-04-01T05:48:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb80ec4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb80806d130 con 0x7fb8081512a0 2024-04-01T05:48:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb807fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb807fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:58320/0 (socket says 172.21.15.64:58320) 2024-04-01T05:48:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb807fff640 1 -- 172.21.15.64:0/1259954224 learned_addr learned my addr 172.21.15.64:0/1259954224 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:51.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb80815b270 con 0x7fb80806bbb0 2024-04-01T05:48:51.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.373+0000 7fb80d1eb640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb80815dcf0 0x7fb8081464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.377+0000 7fb807fff640 1 -- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb80806bbb0 msgr2=0x7fb808150bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-01T05:48:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.377+0000 7fb807fff640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb80806bbb0 0x7fb808150bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.377+0000 7fb807fff640 1 -- 172.21.15.64:0/1259954224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb80815dcf0 msgr2=0x7fb8081464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.377+0000 7fb807fff640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb80815dcf0 0x7fb8081464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.377+0000 7fb807fff640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081469e0 con 0x7fb8081512a0 2024-04-01T05:48:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.382+0000 7fb807fff640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e50 tx=0x7fb800004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.384+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800055020 con 0x7fb8081512a0 2024-04-01T05:48:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.384+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800051470 con 0x7fb8081512a0 2024-04-01T05:48:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.384+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800051820 con 0x7fb8081512a0 2024-04-01T05:48:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.388+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb808150920 con 0x7fb8081512a0 2024-04-01T05:48:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.388+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8080b00b0 con 0x7fb8081512a0 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.394+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb800058020 con 0x7fb8081512a0 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.396+0000 7fb8057fa640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 0x7fb7f406df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.396+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb8000bdf50 con 0x7fb8081512a0 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.397+0000 7fb80c9ea640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 0x7fb7f406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.397+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e8005290 con 0x7fb8081512a0 2024-04-01T05:48:51.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.398+0000 7fb80c9ea640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 0x7fb7f406df50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009670 tx=0x7fb7fc006bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:51.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:51.413+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb80005f030 con 0x7fb8081512a0 2024-04-01T05:48:52.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.063+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7fb7e8005580 con 0x7fb8081512a0 2024-04-01T05:48:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.787+0000 7fb8057fa640 1 -- 172.21.15.64:0/1259954224 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v10132) ==== 144+0+0 (secure 0 0 0) 0x7fb80008cf00 con 0x7fb8081512a0 2024-04-01T05:48:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.794+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 msgr2=0x7fb7f406df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.794+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 0x7fb7f406df50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009670 tx=0x7fb7fc006bd0 comp rx=0 tx=0).stop 2024-04-01T05:48:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.794+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 msgr2=0x7fb8080af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.794+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e50 tx=0x7fb800004a40 comp rx=0 tx=0).stop 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 shutdown_connections 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb7f406ba90 0x7fb7f406df50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb80815dcf0 0x7fb8081464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8081512a0 0x7fb8080af740 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 --2- 172.21.15.64:0/1259954224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb80806bbb0 0x7fb808150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 >> 172.21.15.64:0/1259954224 conn(0x7fb80806b0c0 msgr2=0x7fb8080ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 shutdown_connections 2024-04-01T05:48:52.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:52.798+0000 7fb80ec4e640 1 -- 172.21.15.64:0/1259954224 wait complete. 2024-04-01T05:48:52.861 INFO:tasks.cephfs_test_runner:test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) ... FAIL 2024-04-01T05:48:52.861 INFO:tasks.cephfs_test_runner: 2024-04-01T05:48:52.861 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-01T05:48:52.861 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-01T05:48:52.861 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:AssertionError: 16 != 0 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:Ran 1 test in 20076.784s 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner: 2024-04-01T05:48:52.862 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner:AssertionError: 16 != 0 2024-04-01T05:48:52.863 INFO:tasks.cephfs_test_runner: 2024-04-01T05:48:52.881 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-01T05:48:53.116 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8ac1191bb37341bb9199a6c70ad548cb Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_384f31f3d3aee7f8ba3f60846b8844c049165583/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-01T05:48:53.118 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-01T05:48:53.137 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-01T05:48:53.179 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-01T05:48:53.180 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T05:48:53.180 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:53.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056374,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-01T05:48:53.240 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-01T05:48:53.240 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:53.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056374,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-01T05:48:53.357 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T05:48:53.357 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:53.377 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3299 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-01T05:48:53.377 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-01T05:48:53.378 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-01T05:48:53.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3299 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-01T05:48:53.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.438 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-01T05:48:53.517 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-01T05:48:53.517 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-01T05:48:53.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-01T05:48:53.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-01T05:48:53.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-01T05:48:53.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.765 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-04-01T05:48:53.976 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-01T05:48:53.976 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-01T05:48:53.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:53.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-01T05:48:54.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:54.034 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-01T05:48:54.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-01T05:48:54.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-04-01T05:48:54.282 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-01T05:48:54.291 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-01T05:48:54.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T05:48:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.674+0000 7fb94209d640 1 Processor -- start 2024-04-01T05:48:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.674+0000 7fb94209d640 1 -- start start 2024-04-01T05:48:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.676+0000 7fb94209d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb93c0afee0 0x7fb93c0d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.676+0000 7fb94209d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c072be0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.676+0000 7fb94209d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c0d07a0 con 0x7fb93c0b02c0 2024-04-01T05:48:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.676+0000 7fb94209d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c0d1b60 con 0x7fb93c0afee0 2024-04-01T05:48:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.677+0000 7fb93bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb93c06b7d0 0x7fb93c15d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48376/0 (socket says 172.21.15.64:48376) 2024-04-01T05:48:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.677+0000 7fb93b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb93c0afee0 0x7fb93c0d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.677+0000 7fb93affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb93c0b02c0 0x7fb93c06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:35596/0 (socket says 172.21.15.64:35596) 2024-04-01T05:48:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.677+0000 7fb93bfff640 1 -- 172.21.15.64:0/2046720692 learned_addr learned my addr 172.21.15.64:0/2046720692 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745134320 0 0) 0x7fb93c0d07a0 con 0x7fb93c0b02c0 2024-04-01T05:48:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb924003520 con 0x7fb93c0b02c0 2024-04-01T05:48:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586983565 0 0) 0x7fb93c072be0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb93c0d07a0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2046720692 >> v1:172.21.15.100:6789/0 conn(0x7fb93c0b02c0 legacy=0x7fb93c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2046720692 >> v1:172.21.15.64:6789/0 conn(0x7fb93c06b7d0 legacy=0x7fb93c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.678+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2046720692 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c071e20 con 0x7fb93c0afee0 2024-04-01T05:48:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.680+0000 7fb93b7fe640 1 --2- 172.21.15.64:0/2046720692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c0afee0 0x7fb93c0d0260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0097a0 tx=0x7fb92c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e330dd24ad4d553a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.681+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0315a0 con 0x7fb93c0afee0 2024-04-01T05:48:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.681+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c037070 con 0x7fb93c0afee0 2024-04-01T05:48:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.681+0000 7fb93a7fc640 1 -- 172.21.15.64:0/2046720692 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c031960 con 0x7fb93c0afee0 2024-04-01T05:48:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.683+0000 7fb94209d640 1 -- 172.21.15.64:0/2046720692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c0afee0 msgr2=0x7fb93c0d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.683+0000 7fb94209d640 1 --2- 172.21.15.64:0/2046720692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c0afee0 0x7fb93c0d0260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0097a0 tx=0x7fb92c02b840 comp rx=0 tx=0).stop 2024-04-01T05:48:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.683+0000 7fb94209d640 1 -- 172.21.15.64:0/2046720692 shutdown_connections 2024-04-01T05:48:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.683+0000 7fb94209d640 1 --2- 172.21.15.64:0/2046720692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c0afee0 0x7fb93c0d0260 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.683+0000 7fb94209d640 1 -- 172.21.15.64:0/2046720692 >> 172.21.15.64:0/2046720692 conn(0x7fb93c0d0c10 msgr2=0x7fb93c0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.685+0000 7fb94209d640 1 -- 172.21.15.64:0/2046720692 shutdown_connections 2024-04-01T05:48:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.685+0000 7fb94209d640 1 -- 172.21.15.64:0/2046720692 wait complete. 2024-04-01T05:48:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.689+0000 7fb94209d640 1 Processor -- start 2024-04-01T05:48:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.690+0000 7fb94209d640 1 -- start start 2024-04-01T05:48:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c15e1b0 0x7fb93c148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0726c0 con 0x7fb93c0b02c0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0723e0 con 0x7fb93c15e1b0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb94209d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0cb260 con 0x7fb93c06b7d0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c15e1b0 0x7fb93c148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:48892/0 (socket says 172.21.15.64:48892) 2024-04-01T05:48:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2366081624 learned_addr learned my addr 172.21.15.64:0/2366081624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.693+0000 7fb93affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44498/0 (socket says 172.21.15.64:44498) 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c15e1b0 msgr2=0x7fb93c148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c15e1b0 0x7fb93c148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2366081624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 msgr2=0x7fb93c15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 -- 172.21.15.64:0/2366081624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c1494a0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93affd640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.694+0000 7fb93b7fe640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098c0 tx=0x7fb92c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.695+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c032630 con 0x7fb93c06b7d0 2024-04-01T05:48:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.695+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c0327f0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.695+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c149780 con 0x7fb93c06b7d0 2024-04-01T05:48:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.695+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c032be0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.695+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c15a580 con 0x7fb93c06b7d0 2024-04-01T05:48:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.697+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb92c04d3f0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.697+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c0688b0 con 0x7fb93c06b7d0 2024-04-01T05:48:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.698+0000 7fb923fff640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 0x7fb92806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.699+0000 7fb93affd640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 0x7fb92806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.699+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb92c02f030 con 0x7fb93c06b7d0 2024-04-01T05:48:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.699+0000 7fb93affd640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 0x7fb92806de80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb93c160c40 tx=0x7fb930007cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:54.703+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb92c040070 con 0x7fb93c06b7d0 2024-04-01T05:48:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.232+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb93c0d1200 con 0x7fb92806b9c0 2024-04-01T05:48:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.233+0000 7fb923fff640 1 -- 172.21.15.64:0/2366081624 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13740 (secure 0 0 0) 0x7fb93c0d1200 con 0x7fb92806b9c0 2024-04-01T05:48:55.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T05:48:55.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 msgr2=0x7fb92806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 0x7fb92806de80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb93c160c40 tx=0x7fb930007cf0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 msgr2=0x7fb93c15c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098c0 tx=0x7fb92c0046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 shutdown_connections 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb92806b9c0 0x7fb92806de80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb93c15e1b0 0x7fb93c148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c0b02c0 0x7fb93c15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 --2- 172.21.15.64:0/2366081624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb93c06b7d0 0x7fb93c15c560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 >> 172.21.15.64:0/2366081624 conn(0x7fb93c0d0c10 msgr2=0x7fb93c15fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 shutdown_connections 2024-04-01T05:48:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.237+0000 7fb94209d640 1 -- 172.21.15.64:0/2366081624 wait complete. 2024-04-01T05:48:55.262 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10087,"stamp":"2024-04-01T05:48:54.872712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":20702856,"kb_used_data":8231284,"kb_used_omap":5092,"kb_used_meta":12466395,"kb_avail":734271864,"statfs":{"total":773094113280,"available":751894388736,"internally_reserved":0,"allocated":8428834816,"data_stored":8409692734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5215210,"internal_metadata":12765588502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":504,"apply_latency_ms":504,"commit_latency_ns":504000000,"apply_latency_ns":504000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31649693148,"num_objects":-27917,"num_object_clones":0,"num_object_copies":-55834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27917,"num_whiteouts":0,"num_read":-2015356,"num_read_kb":-141583838,"num_write":-7487989,"num_write_kb":-166648633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1971,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-01T05:48:49.085405+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T05:48:49.085405+0000","last_peered":"2024-04-01T05:48:49.085405+0000","last_clean":"2024-04-01T05:48:49.085405+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T05:48:49.085405+0000","last_undegraded":"2024-04-01T05:48:49.085405+0000","last_fullsized":"2024-04-01T05:48:49.085405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834578873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1848960,"kb_used_data":214132,"kb_used_omap":388,"kb_used_meta":1634427,"kb_avail":92522880,"statfs":{"total":96636764160,"available":94743429120,"internally_reserved":0,"allocated":219271168,"data_stored":218629486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":398295,"internal_metadata":1673653289},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834578874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4703992,"kb_used_data":3096364,"kb_used_omap":406,"kb_used_meta":1607209,"kb_avail":89667848,"statfs":{"total":96636764160,"available":91819876352,"internally_reserved":0,"allocated":3170676736,"data_stored":3164220734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":416539,"internal_metadata":1645782245},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834578875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1947424,"kb_used_data":176148,"kb_used_omap":144,"kb_used_meta":1771119,"kb_avail":92424416,"statfs":{"total":96636764160,"available":94642601984,"internally_reserved":0,"allocated":180375552,"data_stored":179794792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":147804,"internal_metadata":1813626532},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834578872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1805024,"kb_used_data":408,"kb_used_omap":839,"kb_used_meta":1803768,"kb_avail":92566816,"statfs":{"total":96636764160,"available":94788419584,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":859451,"internal_metadata":1847059141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834578834,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4933260,"kb_used_data":3763772,"kb_used_omap":1243,"kb_used_meta":1168228,"kb_avail":89438580,"statfs":{"total":96636764160,"available":91585105920,"internally_reserved":0,"allocated":3854102528,"data_stored":3845655381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1273266,"internal_metadata":1196266062},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":115,"apply_latency_ms":115,"commit_latency_ns":115000000,"apply_latency_ns":115000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834578854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1302176,"kb_used_data":408,"kb_used_omap":315,"kb_used_meta":1301444,"kb_avail":93069664,"statfs":{"total":96636764160,"available":95303335936,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":323406,"internal_metadata":1332678834},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834578848,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1800840,"kb_used_data":124352,"kb_used_omap":1472,"kb_used_meta":1675007,"kb_avail":92571000,"statfs":{"total":96636764160,"available":94792704000,"internally_reserved":0,"allocated":127336448,"data_stored":126819490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507447,"internal_metadata":1715208073},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":111,"apply_latency_ms":111,"commit_latency_ns":111000000,"apply_latency_ns":111000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834578867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2361180,"kb_used_data":855700,"kb_used_omap":282,"kb_used_meta":1505189,"kb_avail":92010660,"statfs":{"total":96636764160,"available":94218915840,"internally_reserved":0,"allocated":876236800,"data_stored":874244337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":289002,"internal_metadata":1541314326},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T05:48:55.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T05:48:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.419+0000 7f018343b640 1 Processor -- start 2024-04-01T05:48:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.419+0000 7f018343b640 1 -- start start 2024-04-01T05:48:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.421+0000 7f018343b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 0x7f017c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.421+0000 7f018343b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f017c0cedd0 0x7f017c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.421+0000 7f018343b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f017c0cf320 con 0x7f017c0d3d00 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.421+0000 7f018343b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f017c15e460 con 0x7f017c0cedd0 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.421+0000 7f018343b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f017c0b9690 con 0x7f017c15e080 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182439640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 0x7f017c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0181c38640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f017c0cedd0 0x7f017c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182c3a640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f017c0d3d00 0x7f017c0d1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:35598/0 (socket says 172.21.15.64:35598) 2024-04-01T05:48:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182c3a640 1 -- 172.21.15.64:0/2546980855 learned_addr learned my addr 172.21.15.64:0/2546980855 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0181437640 1 -- 172.21.15.64:0/2546980855 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52269707 0 0) 0x7f017c0cf320 con 0x7f017c0d3d00 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0181437640 1 -- 172.21.15.64:0/2546980855 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0168003520 con 0x7f017c0d3d00 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182439640 1 -- 172.21.15.64:0/2546980855 >> v2:172.21.15.100:3300/0 conn(0x7f017c0cedd0 msgr2=0x7f017c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182439640 1 --2- 172.21.15.64:0/2546980855 >> v2:172.21.15.100:3300/0 conn(0x7f017c0cedd0 0x7f017c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182439640 1 -- 172.21.15.64:0/2546980855 >> v1:172.21.15.100:6789/0 conn(0x7f017c0d3d00 legacy=0x7f017c0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.422+0000 7f0182439640 1 -- 172.21.15.64:0/2546980855 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c0b9970 con 0x7f017c15e080 2024-04-01T05:48:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.424+0000 7f0182439640 1 --2- 172.21.15.64:0/2546980855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 0x7f017c0b6ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01780097a0 tx=0x7f017802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7563041599ee4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.424+0000 7f0181437640 1 -- 172.21.15.64:0/2546980855 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178035020 con 0x7f017c15e080 2024-04-01T05:48:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.424+0000 7f0181437640 1 -- 172.21.15.64:0/2546980855 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0178031830 con 0x7f017c15e080 2024-04-01T05:48:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.424+0000 7f0181437640 1 -- 172.21.15.64:0/2546980855 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178031c00 con 0x7f017c15e080 2024-04-01T05:48:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.425+0000 7f018343b640 1 -- 172.21.15.64:0/2546980855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 msgr2=0x7f017c0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.425+0000 7f018343b640 1 --2- 172.21.15.64:0/2546980855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 0x7f017c0b6ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01780097a0 tx=0x7f017802b8d0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 -- 172.21.15.64:0/2546980855 shutdown_connections 2024-04-01T05:48:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 --2- 172.21.15.64:0/2546980855 >> v2:172.21.15.100:3300/0 conn(0x7f017c0cedd0 0x7f017c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 --2- 172.21.15.64:0/2546980855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c15e080 0x7f017c0b6ea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 -- 172.21.15.64:0/2546980855 >> 172.21.15.64:0/2546980855 conn(0x7f017c071ad0 msgr2=0x7f017c071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 -- 172.21.15.64:0/2546980855 shutdown_connections 2024-04-01T05:48:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.426+0000 7f018343b640 1 -- 172.21.15.64:0/2546980855 wait complete. 2024-04-01T05:48:55.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.429+0000 7f018343b640 1 Processor -- start 2024-04-01T05:48:55.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.429+0000 7f018343b640 1 -- start start 2024-04-01T05:48:55.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c15e080 0x7f017c1f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f017c0d4190 con 0x7f017c15e080 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f017c0b3500 con 0x7f017c0d3d00 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.431+0000 7f018343b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f017c0d4310 con 0x7f017c0cedd0 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0182439640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0182439640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:48914/0 (socket says 172.21.15.64:48914) 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:33352/0 (socket says 172.21.15.64:33352) 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0182439640 1 -- 172.21.15.64:0/2266981594 learned_addr learned my addr 172.21.15.64:0/2266981594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0182c3a640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c15e080 0x7f017c1f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 -- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 msgr2=0x7f017c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 -- 172.21.15.64:0/2266981594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c15e080 msgr2=0x7f017c1f6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c15e080 0x7f017c1f6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0181c38640 1 -- 172.21.15.64:0/2266981594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c15fd40 con 0x7f017c0d3d00 2024-04-01T05:48:55.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.432+0000 7f0182439640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:55.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f0181c38640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e880 tx=0x7f016c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:55.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c017070 con 0x7f017c0d3d00 2024-04-01T05:48:55.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016c00c940 con 0x7f017c0d3d00 2024-04-01T05:48:55.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c00cd30 con 0x7f017c0d3d00 2024-04-01T05:48:55.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c160020 con 0x7f017c0d3d00 2024-04-01T05:48:55.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.433+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c14ce80 con 0x7f017c0d3d00 2024-04-01T05:48:55.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.435+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c04f4b0 con 0x7f017c0d3d00 2024-04-01T05:48:55.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.436+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f016c01a070 con 0x7f017c0d3d00 2024-04-01T05:48:55.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.437+0000 7f0172ffd640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 0x7f016006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:55.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.437+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f016c08a740 con 0x7f017c0d3d00 2024-04-01T05:48:55.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.437+0000 7f0182439640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 0x7f016006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:55.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.438+0000 7f0182439640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 0x7f016006de50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f017802b910 tx=0x7f017803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.442+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f016c059090 con 0x7f017c0d3d00 2024-04-01T05:48:55.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.973+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f017c0cb2c0 con 0x7f016006b990 2024-04-01T05:48:55.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.974+0000 7f0172ffd640 1 -- 172.21.15.64:0/2266981594 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13740 (secure 0 0 0) 0x7f017804b5c0 con 0x7f016006b990 2024-04-01T05:48:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:55.975 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T05:48:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 msgr2=0x7f016006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 0x7f016006de50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f017802b910 tx=0x7f017803c040 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 msgr2=0x7f017c15f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e880 tx=0x7f016c00ed40 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 shutdown_connections 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f016006b990 0x7f016006de50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c15e080 0x7f017c1f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017c0d3d00 0x7f017c15f4f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 --2- 172.21.15.64:0/2266981594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f017c0cedd0 0x7f017c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 >> 172.21.15.64:0/2266981594 conn(0x7f017c071ad0 msgr2=0x7f017c15d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:55.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 shutdown_connections 2024-04-01T05:48:55.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:55.978+0000 7f018343b640 1 -- 172.21.15.64:0/2266981594 wait complete. 2024-04-01T05:48:56.003 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10087,"stamp":"2024-04-01T05:48:54.872712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":20702856,"kb_used_data":8231284,"kb_used_omap":5092,"kb_used_meta":12466395,"kb_avail":734271864,"statfs":{"total":773094113280,"available":751894388736,"internally_reserved":0,"allocated":8428834816,"data_stored":8409692734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5215210,"internal_metadata":12765588502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":504,"apply_latency_ms":504,"commit_latency_ns":504000000,"apply_latency_ns":504000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31649693148,"num_objects":-27917,"num_object_clones":0,"num_object_copies":-55834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27917,"num_whiteouts":0,"num_read":-2015356,"num_read_kb":-141583838,"num_write":-7487989,"num_write_kb":-166648633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1971,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-01T05:48:49.085405+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T05:48:49.085405+0000","last_peered":"2024-04-01T05:48:49.085405+0000","last_clean":"2024-04-01T05:48:49.085405+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T05:48:49.085405+0000","last_undegraded":"2024-04-01T05:48:49.085405+0000","last_fullsized":"2024-04-01T05:48:49.085405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834578873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1848960,"kb_used_data":214132,"kb_used_omap":388,"kb_used_meta":1634427,"kb_avail":92522880,"statfs":{"total":96636764160,"available":94743429120,"internally_reserved":0,"allocated":219271168,"data_stored":218629486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":398295,"internal_metadata":1673653289},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834578874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4703992,"kb_used_data":3096364,"kb_used_omap":406,"kb_used_meta":1607209,"kb_avail":89667848,"statfs":{"total":96636764160,"available":91819876352,"internally_reserved":0,"allocated":3170676736,"data_stored":3164220734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":416539,"internal_metadata":1645782245},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834578875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1947424,"kb_used_data":176148,"kb_used_omap":144,"kb_used_meta":1771119,"kb_avail":92424416,"statfs":{"total":96636764160,"available":94642601984,"internally_reserved":0,"allocated":180375552,"data_stored":179794792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":147804,"internal_metadata":1813626532},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834578872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1805024,"kb_used_data":408,"kb_used_omap":839,"kb_used_meta":1803768,"kb_avail":92566816,"statfs":{"total":96636764160,"available":94788419584,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":859451,"internal_metadata":1847059141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834578834,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4933260,"kb_used_data":3763772,"kb_used_omap":1243,"kb_used_meta":1168228,"kb_avail":89438580,"statfs":{"total":96636764160,"available":91585105920,"internally_reserved":0,"allocated":3854102528,"data_stored":3845655381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1273266,"internal_metadata":1196266062},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":115,"apply_latency_ms":115,"commit_latency_ns":115000000,"apply_latency_ns":115000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834578854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1302176,"kb_used_data":408,"kb_used_omap":315,"kb_used_meta":1301444,"kb_avail":93069664,"statfs":{"total":96636764160,"available":95303335936,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":323406,"internal_metadata":1332678834},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834578848,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1800840,"kb_used_data":124352,"kb_used_omap":1472,"kb_used_meta":1675007,"kb_avail":92571000,"statfs":{"total":96636764160,"available":94792704000,"internally_reserved":0,"allocated":127336448,"data_stored":126819490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507447,"internal_metadata":1715208073},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":111,"apply_latency_ms":111,"commit_latency_ns":111000000,"apply_latency_ns":111000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834578867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2361180,"kb_used_data":855700,"kb_used_omap":282,"kb_used_meta":1505189,"kb_avail":92010660,"statfs":{"total":96636764160,"available":94218915840,"internally_reserved":0,"allocated":876236800,"data_stored":874244337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":289002,"internal_metadata":1541314326},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T05:48:56.004 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-01T05:48:56.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T05:48:56.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.272+0000 7f9b1667c640 1 Processor -- start 2024-04-01T05:48:56.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.272+0000 7f9b1667c640 1 -- start start 2024-04-01T05:48:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.274+0000 7f9b1667c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 0x7f9b100ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.274+0000 7f9b1667c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1006b570 con 0x7f9b1006b0c0 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.274+0000 7f9b1667c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1006b7f0 con 0x7f9b1015eb90 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.274+0000 7f9b1667c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1006bad0 con 0x7f9b100cc770 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.274+0000 7f9b0f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9b1006b0c0 0x7f9b1015ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48378/0 (socket says 172.21.15.64:48378) 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b0f7fe640 1 -- 172.21.15.64:0/2613743132 learned_addr learned my addr 172.21.15.64:0/2613743132 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b14c19640 1 --2- 172.21.15.64:0/2613743132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 0x7f9b100ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b14c19640 1 -- 172.21.15.64:0/2613743132 >> v1:172.21.15.100:6789/0 conn(0x7f9b1015eb90 legacy=0x7f9b10162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b14c19640 1 -- 172.21.15.64:0/2613743132 >> v1:172.21.15.64:6789/0 conn(0x7f9b1006b0c0 legacy=0x7f9b1015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b0effd640 1 -- 172.21.15.64:0/2613743132 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164546504 0 0) 0x7f9b1006b7f0 con 0x7f9b1015eb90 2024-04-01T05:48:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.275+0000 7f9b14c19640 1 -- 172.21.15.64:0/2613743132 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b10161d30 con 0x7f9b100cc770 2024-04-01T05:48:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.277+0000 7f9b14c19640 1 --2- 172.21.15.64:0/2613743132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 0x7f9b100ccc20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009dc0 tx=0x7f9b0802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab0a16861a73d15e server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.277+0000 7f9b0effd640 1 -- 172.21.15.64:0/2613743132 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08039070 con 0x7f9b100cc770 2024-04-01T05:48:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.278+0000 7f9b0effd640 1 -- 172.21.15.64:0/2613743132 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0802ca80 con 0x7f9b100cc770 2024-04-01T05:48:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.278+0000 7f9b0effd640 1 -- 172.21.15.64:0/2613743132 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0802ce50 con 0x7f9b100cc770 2024-04-01T05:48:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 -- 172.21.15.64:0/2613743132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 msgr2=0x7f9b100ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2613743132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 0x7f9b100ccc20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009dc0 tx=0x7f9b0802bd50 comp rx=0 tx=0).stop 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 -- 172.21.15.64:0/2613743132 shutdown_connections 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2613743132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b100cc770 0x7f9b100ccc20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 -- 172.21.15.64:0/2613743132 >> 172.21.15.64:0/2613743132 conn(0x7f9b100d0be0 msgr2=0x7f9b100d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 -- 172.21.15.64:0/2613743132 shutdown_connections 2024-04-01T05:48:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.279+0000 7f9b1667c640 1 -- 172.21.15.64:0/2613743132 wait complete. 2024-04-01T05:48:56.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.282+0000 7f9b1667c640 1 Processor -- start 2024-04-01T05:48:56.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.282+0000 7f9b1667c640 1 -- start start 2024-04-01T05:48:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.284+0000 7f9b1667c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 0x7f9b100a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.284+0000 7f9b1667c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 0x7f9b100b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.284+0000 7f9b1667c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 0x7f9b100a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.284+0000 7f9b1667c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1015f990 con 0x7f9b100cc770 2024-04-01T05:48:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.284+0000 7f9b1667c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b100cd160 con 0x7f9b1006b0c0 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b1667c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1015fb10 con 0x7f9b1015eb90 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 0x7f9b100b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 0x7f9b100b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44514/0 (socket says 172.21.15.64:44514) 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 0x7f9b100a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b0f7fe640 1 -- 172.21.15.64:0/2411837381 learned_addr learned my addr 172.21.15.64:0/2411837381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b14c19640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 0x7f9b100a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:56.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.285+0000 7f9b0ffff640 1 -- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 msgr2=0x7f9b100a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0ffff640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 0x7f9b100a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0ffff640 1 -- 172.21.15.64:0/2411837381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 msgr2=0x7f9b100b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0ffff640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 0x7f9b100b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0ffff640 1 -- 172.21.15.64:0/2411837381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100b5ee0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b14c19640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 0x7f9b100a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0ffff640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 0x7f9b100a1790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000c410 tx=0x7f9b0000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00017070 con 0x7f9b1006b0c0 2024-04-01T05:48:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b000035f0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.286+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100b61c0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.287+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000039e0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.287+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b100b6780 con 0x7f9b1006b0c0 2024-04-01T05:48:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.289+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b100688b0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.289+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f9b0001a070 con 0x7f9b1006b0c0 2024-04-01T05:48:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.290+0000 7f9b0cff9640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 0x7f9afc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.291+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9b000898c0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.291+0000 7f9b0f7fe640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 0x7f9afc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.291+0000 7f9b0f7fe640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 0x7f9afc06de50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9b1006fd60 tx=0x7f9b04000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.295+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f9b000580e0 con 0x7f9b1006b0c0 2024-04-01T05:48:56.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.814+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b100cb230 con 0x7f9afc06b990 2024-04-01T05:48:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.815+0000 7f9b0cff9640 1 -- 172.21.15.64:0/2411837381 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13740 (secure 0 0 0) 0x7f9b100cb230 con 0x7f9afc06b990 2024-04-01T05:48:56.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T05:48:56.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 msgr2=0x7f9afc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 0x7f9afc06de50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9b1006fd60 tx=0x7f9b04000bc0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 msgr2=0x7f9b100a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 0x7f9b100a1790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000c410 tx=0x7f9b0000c8d0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 shutdown_connections 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f9afc06b990 0x7f9afc06de50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b1015eb90 0x7f9b100a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b100cc770 0x7f9b100b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 --2- 172.21.15.64:0/2411837381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1006b0c0 0x7f9b100a1790 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 >> 172.21.15.64:0/2411837381 conn(0x7f9b100d0be0 msgr2=0x7f9b100d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:56.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 shutdown_connections 2024-04-01T05:48:56.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:56.819+0000 7f9b1667c640 1 -- 172.21.15.64:0/2411837381 wait complete. 2024-04-01T05:48:56.843 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10087,"stamp":"2024-04-01T05:48:54.872712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":20702856,"kb_used_data":8231284,"kb_used_omap":5092,"kb_used_meta":12466395,"kb_avail":734271864,"statfs":{"total":773094113280,"available":751894388736,"internally_reserved":0,"allocated":8428834816,"data_stored":8409692734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5215210,"internal_metadata":12765588502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":504,"apply_latency_ms":504,"commit_latency_ns":504000000,"apply_latency_ns":504000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31649693148,"num_objects":-27917,"num_object_clones":0,"num_object_copies":-55834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27917,"num_whiteouts":0,"num_read":-2015356,"num_read_kb":-141583838,"num_write":-7487989,"num_write_kb":-166648633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1971,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-01T05:48:49.085405+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T05:48:49.085405+0000","last_peered":"2024-04-01T05:48:49.085405+0000","last_clean":"2024-04-01T05:48:49.085405+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T05:48:49.085405+0000","last_undegraded":"2024-04-01T05:48:49.085405+0000","last_fullsized":"2024-04-01T05:48:49.085405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834578873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1848960,"kb_used_data":214132,"kb_used_omap":388,"kb_used_meta":1634427,"kb_avail":92522880,"statfs":{"total":96636764160,"available":94743429120,"internally_reserved":0,"allocated":219271168,"data_stored":218629486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":398295,"internal_metadata":1673653289},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834578874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4703992,"kb_used_data":3096364,"kb_used_omap":406,"kb_used_meta":1607209,"kb_avail":89667848,"statfs":{"total":96636764160,"available":91819876352,"internally_reserved":0,"allocated":3170676736,"data_stored":3164220734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":416539,"internal_metadata":1645782245},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834578875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1947424,"kb_used_data":176148,"kb_used_omap":144,"kb_used_meta":1771119,"kb_avail":92424416,"statfs":{"total":96636764160,"available":94642601984,"internally_reserved":0,"allocated":180375552,"data_stored":179794792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":147804,"internal_metadata":1813626532},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834578872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1805024,"kb_used_data":408,"kb_used_omap":839,"kb_used_meta":1803768,"kb_avail":92566816,"statfs":{"total":96636764160,"available":94788419584,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":859451,"internal_metadata":1847059141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834578834,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4933260,"kb_used_data":3763772,"kb_used_omap":1243,"kb_used_meta":1168228,"kb_avail":89438580,"statfs":{"total":96636764160,"available":91585105920,"internally_reserved":0,"allocated":3854102528,"data_stored":3845655381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1273266,"internal_metadata":1196266062},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":115,"apply_latency_ms":115,"commit_latency_ns":115000000,"apply_latency_ns":115000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834578854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1302176,"kb_used_data":408,"kb_used_omap":315,"kb_used_meta":1301444,"kb_avail":93069664,"statfs":{"total":96636764160,"available":95303335936,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":323406,"internal_metadata":1332678834},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834578848,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1800840,"kb_used_data":124352,"kb_used_omap":1472,"kb_used_meta":1675007,"kb_avail":92571000,"statfs":{"total":96636764160,"available":94792704000,"internally_reserved":0,"allocated":127336448,"data_stored":126819490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507447,"internal_metadata":1715208073},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":111,"apply_latency_ms":111,"commit_latency_ns":111000000,"apply_latency_ns":111000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834578867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2361180,"kb_used_data":855700,"kb_used_omap":282,"kb_used_meta":1505189,"kb_avail":92010660,"statfs":{"total":96636764160,"available":94218915840,"internally_reserved":0,"allocated":876236800,"data_stored":874244337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":289002,"internal_metadata":1541314326},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T05:48:56.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-01T05:48:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.119+0000 7f2ccbe47640 1 Processor -- start 2024-04-01T05:48:57.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.119+0000 7f2ccbe47640 1 -- start start 2024-04-01T05:48:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cc40d09c0 0x7f2cc40c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d01b0 0x7f2cc40d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 0x7f2cc40c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc40b4fb0 con 0x7f2cc40d12e0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc40b53d0 con 0x7f2cc40d09c0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2ccbe47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc40b56b0 con 0x7f2cc40d01b0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.121+0000 7f2cc9be3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cc40d09c0 0x7f2cc40c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cc40d09c0 0x7f2cc40c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:33372/0 (socket says 172.21.15.64:33372) 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 -- 172.21.15.64:0/3812417820 learned_addr learned my addr 172.21.15.64:0/3812417820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc93e2640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d01b0 0x7f2cc40d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cca3e4640 1 --2- 172.21.15.64:0/3812417820 >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 0x7f2cc40c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 -- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d01b0 msgr2=0x7f2cc40d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d01b0 0x7f2cc40d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 -- 172.21.15.64:0/3812417820 >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 msgr2=0x7f2cc40c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/3812417820 >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 0x7f2cc40c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cc9be3640 1 -- 172.21.15.64:0/3812417820 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc40cf9b0 con 0x7f2cc40d09c0 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.122+0000 7f2cca3e4640 1 --2- 172.21.15.64:0/3812417820 >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 0x7f2cc40c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.123+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40c73c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40d3f10 tx=0x7f2cc002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791fd036cac01963 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.123+0000 7f2cc8be1640 1 -- 172.21.15.64:0/3812417820 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc002e070 con 0x7f2cc40d09c0 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.123+0000 7f2cc8be1640 1 -- 172.21.15.64:0/3812417820 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc0037070 con 0x7f2cc40d09c0 2024-04-01T05:48:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.123+0000 7f2cc8be1640 1 -- 172.21.15.64:0/3812417820 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0031a00 con 0x7f2cc40d09c0 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 -- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d09c0 msgr2=0x7f2cc40c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40c73c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40d3f10 tx=0x7f2cc002b880 comp rx=0 tx=0).stop 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 -- 172.21.15.64:0/3812417820 shutdown_connections 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/3812417820 >> v2:172.21.15.64:3300/0 conn(0x7f2cc40d12e0 0x7f2cc40c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d01b0 0x7f2cc40d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/3812417820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40c73c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 -- 172.21.15.64:0/3812417820 >> 172.21.15.64:0/3812417820 conn(0x7f2cc409d290 msgr2=0x7f2cc40d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 -- 172.21.15.64:0/3812417820 shutdown_connections 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.125+0000 7f2ccbe47640 1 -- 172.21.15.64:0/3812417820 wait complete. 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.128+0000 7f2ccbe47640 1 Processor -- start 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.128+0000 7f2ccbe47640 1 -- start start 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.130+0000 7f2ccbe47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.130+0000 7f2ccbe47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.130+0000 7f2ccbe47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 0x7f2cc409fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2ccbe47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc40d1770 con 0x7f2cc40d09c0 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2ccbe47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc40d0630 con 0x7f2cc40d01b0 2024-04-01T05:48:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2ccbe47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc40d18f0 con 0x7f2cc40d12e0 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc93e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:33382/0 (socket says 172.21.15.64:33382) 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 -- 172.21.15.64:0/4104974365 learned_addr learned my addr 172.21.15.64:0/4104974365 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cca3e4640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 0x7f2cc409fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 -- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 msgr2=0x7f2cc409fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 0x7f2cc409fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 -- 172.21.15.64:0/4104974365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40d09c0 msgr2=0x7f2cc40a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cc9be3640 1 -- 172.21.15.64:0/4104974365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc41e1720 con 0x7f2cc40d01b0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.131+0000 7f2cca3e4640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 0x7f2cc409fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2cc9be3640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2cc002b8c0 tx=0x7f2cc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0032670 con 0x7f2cc40d01b0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc0032810 con 0x7f2cc40d01b0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc40b1b50 con 0x7f2cc40d01b0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0032c00 con 0x7f2cc40d01b0 2024-04-01T05:48:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.132+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc40b2110 con 0x7f2cc40d01b0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.135+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2cc004c060 con 0x7f2cc40d01b0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.135+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc40681a0 con 0x7f2cc40d01b0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.136+0000 7f2cba7fc640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 0x7f2ca806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.136+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f2cc00abfc0 con 0x7f2cc40d01b0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.136+0000 7f2cc93e2640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 0x7f2ca806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.137+0000 7f2cc93e2640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 0x7f2ca806de50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40c7020 tx=0x7f2cb4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.141+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2cc0040070 con 0x7f2cc40d01b0 2024-04-01T05:48:57.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.673+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cc406c0d0 con 0x7f2cc40d01b0 2024-04-01T05:48:57.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.674+0000 7f2cba7fc640 1 -- 172.21.15.64:0/4104974365 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12998 (secure 0 0 0) 0x7f2cc007a910 con 0x7f2cc40d01b0 2024-04-01T05:48:57.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:48:57.675 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"05d5f08b-c3c0-4c90-b927-5e55cbe6aed5","created":"2024-04-01T00:13:22.526769+0000","modified":"2024-04-01T05:48:49.065309+0000","last_up_change":"2024-04-01T00:13:33.824114+0000","last_in_change":"2024-04-01T00:13:24.723519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-01T00:13:34.778476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e752169-6147-48da-8dd8-9a0e36056013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6801","nonce":3875769055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6803","nonce":3875769055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6807","nonce":3875769055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3875769055},{"type":"v1","addr":"172.21.15.64:6805","nonce":3875769055}]},"public_addr":"172.21.15.64:6801/3875769055","cluster_addr":"172.21.15.64:6803/3875769055","heartbeat_back_addr":"172.21.15.64:6807/3875769055","heartbeat_front_addr":"172.21.15.64:6805/3875769055","state":["exists","up"]},{"osd":1,"uuid":"ade989fe-45b3-4539-a94f-d87f9387130b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6817","nonce":199266641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6819","nonce":199266641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6823","nonce":199266641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":199266641},{"type":"v1","addr":"172.21.15.64:6821","nonce":199266641}]},"public_addr":"172.21.15.64:6817/199266641","cluster_addr":"172.21.15.64:6819/199266641","heartbeat_back_addr":"172.21.15.64:6823/199266641","heartbeat_front_addr":"172.21.15.64:6821/199266641","state":["exists","up"]},{"osd":2,"uuid":"ea0a8972-b335-43d1-b796-294fe768a068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6825","nonce":2000381811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6827","nonce":2000381811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6831","nonce":2000381811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2000381811},{"type":"v1","addr":"172.21.15.64:6829","nonce":2000381811}]},"public_addr":"172.21.15.64:6825/2000381811","cluster_addr":"172.21.15.64:6827/2000381811","heartbeat_back_addr":"172.21.15.64:6831/2000381811","heartbeat_front_addr":"172.21.15.64:6829/2000381811","state":["exists","up"]},{"osd":3,"uuid":"3f21b4bf-b9dd-4772-a5de-ea0819be0b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6809","nonce":2132154805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6811","nonce":2132154805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6815","nonce":2132154805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2132154805},{"type":"v1","addr":"172.21.15.64:6813","nonce":2132154805}]},"public_addr":"172.21.15.64:6809/2132154805","cluster_addr":"172.21.15.64:6811/2132154805","heartbeat_back_addr":"172.21.15.64:6815/2132154805","heartbeat_front_addr":"172.21.15.64:6813/2132154805","state":["exists","up"]},{"osd":4,"uuid":"0f18736e-7c9c-424e-b829-ab17c5a8b64d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6817","nonce":3826994341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6819","nonce":3826994341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6823","nonce":3826994341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3826994341},{"type":"v1","addr":"172.21.15.100:6821","nonce":3826994341}]},"public_addr":"172.21.15.100:6817/3826994341","cluster_addr":"172.21.15.100:6819/3826994341","heartbeat_back_addr":"172.21.15.100:6823/3826994341","heartbeat_front_addr":"172.21.15.100:6821/3826994341","state":["exists","up"]},{"osd":5,"uuid":"9c19a862-8d4e-4a8d-9efb-c44c4f905fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6801","nonce":64687588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6803","nonce":64687588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6807","nonce":64687588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":64687588},{"type":"v1","addr":"172.21.15.100:6805","nonce":64687588}]},"public_addr":"172.21.15.100:6801/64687588","cluster_addr":"172.21.15.100:6803/64687588","heartbeat_back_addr":"172.21.15.100:6807/64687588","heartbeat_front_addr":"172.21.15.100:6805/64687588","state":["exists","up"]},{"osd":6,"uuid":"d7a44ad8-6b22-4573-a0da-8fb58d1c82a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6809","nonce":3251071123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6811","nonce":3251071123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6815","nonce":3251071123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3251071123},{"type":"v1","addr":"172.21.15.100:6813","nonce":3251071123}]},"public_addr":"172.21.15.100:6809/3251071123","cluster_addr":"172.21.15.100:6811/3251071123","heartbeat_back_addr":"172.21.15.100:6815/3251071123","heartbeat_front_addr":"172.21.15.100:6813/3251071123","state":["exists","up"]},{"osd":7,"uuid":"f3eb2914-1b89-4a42-8a43-b21819229c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6825","nonce":2776403688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6827","nonce":2776403688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6831","nonce":2776403688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2776403688},{"type":"v1","addr":"172.21.15.100:6829","nonce":2776403688}]},"public_addr":"172.21.15.100:6825/2776403688","cluster_addr":"172.21.15.100:6827/2776403688","heartbeat_back_addr":"172.21.15.100:6831/2776403688","heartbeat_front_addr":"172.21.15.100:6829/2776403688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.804845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.845002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.868121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.800876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.856194+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.884202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.896973+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-01T00:13:29.932679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6837/1975239378":"2024-04-02T05:48:45.409596+0000","172.21.15.100:6835/1975239378":"2024-04-02T05:48:45.409596+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-01T05:48:57.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 msgr2=0x7f2ca806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 0x7f2ca806de50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40c7020 tx=0x7f2cb4009190 comp rx=0 tx=0).stop 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 msgr2=0x7f2cc40b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2cc002b8c0 tx=0x7f2cc00046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 shutdown_connections 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2ca806b990 0x7f2ca806de50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cc40d12e0 0x7f2cc409fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40d09c0 0x7f2cc40a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 --2- 172.21.15.64:0/4104974365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cc40d01b0 0x7f2cc40b2bf0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 >> 172.21.15.64:0/4104974365 conn(0x7f2cc409d290 msgr2=0x7f2cc40d3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 shutdown_connections 2024-04-01T05:48:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:57.678+0000 7f2ccbe47640 1 -- 172.21.15.64:0/4104974365 wait complete. 2024-04-01T05:48:58.704 INFO:tasks.ceph:Scrubbing osd.0 2024-04-01T05:48:58.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:48:58.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.994+0000 7f1a82272640 1 Processor -- start 2024-04-01T05:48:58.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.995+0000 7f1a82272640 1 -- start start 2024-04-01T05:48:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c5f0 0x7f1a7c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a7c15c9d0 0x7f1a7c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a7c071680 0x7f1a7c071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7c072420 con 0x7f1a7c071680 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7c072700 con 0x7f1a7c15c9d0 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a82272640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c0cee50 con 0x7f1a7c15c5f0 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a8080f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a7c071680 0x7f1a7c071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a8080f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a7c071680 0x7f1a7c071b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44532/0 (socket says 172.21.15.64:44532) 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a8080f640 1 -- 172.21.15.64:0/4255436717 learned_addr learned my addr 172.21.15.64:0/4255436717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a7b7fe640 1 --2- 172.21.15.64:0/4255436717 >> v2:172.21.15.100:3300/0 conn(0x7f1a7c15c9d0 0x7f1a7c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.997+0000 7f1a7bfff640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c5f0 0x7f1a7c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:58.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.998+0000 7f1a8080f640 1 -- 172.21.15.64:0/4255436717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c5f0 msgr2=0x7f1a7c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:58.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.998+0000 7f1a8080f640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c5f0 0x7f1a7c15d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.998+0000 7f1a8080f640 1 -- 172.21.15.64:0/4255436717 >> v2:172.21.15.100:3300/0 conn(0x7f1a7c15c9d0 msgr2=0x7f1a7c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.998+0000 7f1a8080f640 1 --2- 172.21.15.64:0/4255436717 >> v2:172.21.15.100:3300/0 conn(0x7f1a7c15c9d0 0x7f1a7c15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.998+0000 7f1a8080f640 1 -- 172.21.15.64:0/4255436717 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c0cf130 con 0x7f1a7c071680 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.999+0000 7f1a8080f640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c071b10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c06d040 tx=0x7f1a7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e825f4288714081 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.999+0000 7f1a7affd640 1 -- 172.21.15.64:0/4255436717 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7402ca70 con 0x7f1a7c071680 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.999+0000 7f1a7affd640 1 -- 172.21.15.64:0/4255436717 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7402cc10 con 0x7f1a7c071680 2024-04-01T05:48:59.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:58.999+0000 7f1a7affd640 1 -- 172.21.15.64:0/4255436717 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a740343f0 con 0x7f1a7c071680 2024-04-01T05:48:59.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.000+0000 7f1a82272640 1 -- 172.21.15.64:0/4255436717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 msgr2=0x7f1a7c071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.000+0000 7f1a82272640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c071b10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c06d040 tx=0x7f1a7402bed0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 -- 172.21.15.64:0/4255436717 shutdown_connections 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c071b10 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 --2- 172.21.15.64:0/4255436717 >> v2:172.21.15.100:3300/0 conn(0x7f1a7c15c9d0 0x7f1a7c15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 --2- 172.21.15.64:0/4255436717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c5f0 0x7f1a7c15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 -- 172.21.15.64:0/4255436717 >> 172.21.15.64:0/4255436717 conn(0x7f1a7c0d03c0 msgr2=0x7f1a7c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 -- 172.21.15.64:0/4255436717 shutdown_connections 2024-04-01T05:48:59.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.001+0000 7f1a82272640 1 -- 172.21.15.64:0/4255436717 wait complete. 2024-04-01T05:48:59.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.004+0000 7f1a82272640 1 Processor -- start 2024-04-01T05:48:59.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.004+0000 7f1a82272640 1 -- start start 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a7c15c5f0 0x7f1a7c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c9d0 0x7f1a7c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0721d0 con 0x7f1a7c071680 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c072050 con 0x7f1a7c15c5f0 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a82272640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c0cea70 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.006+0000 7f1a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44540/0 (socket says 172.21.15.64:44540) 2024-04-01T05:48:59.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a7bfff640 1 -- 172.21.15.64:0/3402724039 learned_addr learned my addr 172.21.15.64:0/3402724039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:59.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a7b7fe640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a7c15c5f0 0x7f1a7c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c9d0 0x7f1a7c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 -- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a7c15c5f0 msgr2=0x7f1a7c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a7c15c5f0 0x7f1a7c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 -- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 msgr2=0x7f1a7c142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c071680 0x7f1a7c142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c1431b0 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.007+0000 7f1a8080f640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c9d0 0x7f1a7c15afe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0b2ff0 tx=0x7f1a74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.008+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74043020 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.008+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7403e450 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.008+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c156220 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.008+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74009510 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.008+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1567e0 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.009+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a7c0688b0 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.010+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1a74046020 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a78ff9640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a6806b990 0x7f1a6806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f1a740acae0 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a7bfff640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a6806b990 0x7f1a6806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a78ff9640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a68071850 0x7f1a68073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a7403be50 con 0x7f1a68071850 2024-04-01T05:48:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a740b12a0 con 0x7f1a7c15c9d0 2024-04-01T05:48:59.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.011+0000 7f1a7b7fe640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a68071850 0x7f1a68073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.012+0000 7f1a7bfff640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a6806b990 0x7f1a6806de50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009c00 tx=0x7f1a6c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.012+0000 7f1a7b7fe640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a68071850 0x7f1a68073c70 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.014+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== osd.0 v2:172.21.15.64:6800/3875769055 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a7403be50 con 0x7f1a68071850 2024-04-01T05:48:59.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.053+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 --> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1a7c0b0940 con 0x7f1a68071850 2024-04-01T05:48:59.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.053+0000 7f1a78ff9640 1 -- 172.21.15.64:0/3402724039 <== osd.0 v2:172.21.15.64:6800/3875769055 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1a7c0b0940 con 0x7f1a68071850 2024-04-01T05:48:59.054 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:48:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:48:59.054 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:48:59.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a68071850 msgr2=0x7f1a68073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.64:6800/3875769055,v1:172.21.15.64:6801/3875769055] conn(0x7f1a68071850 0x7f1a68073c70 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a6806b990 msgr2=0x7f1a6806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f1a6806b990 0x7f1a6806de50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009c00 tx=0x7f1a6c009190 comp rx=0 tx=0).stop 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c9d0 msgr2=0x7f1a7c15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 --2- 172.21.15.64:0/3402724039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a7c15c9d0 0x7f1a7c15afe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0b2ff0 tx=0x7f1a74004a40 comp rx=0 tx=0).stop 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a8080f640 1 -- 172.21.15.64:0/3402724039 reap_dead start 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 shutdown_connections 2024-04-01T05:48:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.054+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 >> 172.21.15.64:0/3402724039 conn(0x7f1a7c0d03c0 msgr2=0x7f1a7c0b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.055+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 shutdown_connections 2024-04-01T05:48:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.055+0000 7f1a82272640 1 -- 172.21.15.64:0/3402724039 wait complete. 2024-04-01T05:48:59.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-01T05:48:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.356+0000 7f2cb32c9640 1 Processor -- start 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.357+0000 7f2cb32c9640 1 -- start start 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb32c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cac06ff70 0x7f2cac071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb32c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb32c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac0d09b0 con 0x7f2cac0cbca0 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb32c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0722a0 con 0x7f2cac06ff70 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb32c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac072580 con 0x7f2cac070350 2024-04-01T05:48:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb1866640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2cac0cbca0 0x7f2cac06b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48392/0 (socket says 172.21.15.64:48392) 2024-04-01T05:48:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb1866640 1 -- 172.21.15.64:0/1801497305 learned_addr learned my addr 172.21.15.64:0/1801497305 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:48:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb1065640 1 --2- 172.21.15.64:0/1801497305 >> v2:172.21.15.100:3300/0 conn(0x7f2cac06ff70 0x7f2cac071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.358+0000 7f2cb0864640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2ca3fff640 1 -- 172.21.15.64:0/1801497305 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657738599 0 0) 0x7f2cac0d09b0 con 0x7f2cac0cbca0 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2ca3fff640 1 -- 172.21.15.64:0/1801497305 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c98003520 con 0x7f2cac0cbca0 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2cb1065640 1 -- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 msgr2=0x7f2cac0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2cb1065640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2cb1065640 1 -- 172.21.15.64:0/1801497305 >> v1:172.21.15.64:6789/0 conn(0x7f2cac0cbca0 legacy=0x7f2cac06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2cb1065640 1 -- 172.21.15.64:0/1801497305 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac072860 con 0x7f2cac06ff70 2024-04-01T05:48:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.359+0000 7f2cb0864640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:48:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.361+0000 7f2cb1065640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac071cc0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80097a0 tx=0x7f2ca802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27eeff93d7df0765 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.361+0000 7f2ca3fff640 1 -- 172.21.15.64:0/1801497305 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80315a0 con 0x7f2cac06ff70 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.361+0000 7f2ca3fff640 1 -- 172.21.15.64:0/1801497305 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca8037070 con 0x7f2cac06ff70 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.361+0000 7f2ca3fff640 1 -- 172.21.15.64:0/1801497305 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8031960 con 0x7f2cac06ff70 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 -- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 msgr2=0x7f2cac071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac071cc0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80097a0 tx=0x7f2ca802b840 comp rx=0 tx=0).stop 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 -- 172.21.15.64:0/1801497305 shutdown_connections 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/1801497305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac071cc0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 -- 172.21.15.64:0/1801497305 >> 172.21.15.64:0/1801497305 conn(0x7f2cac0d0c70 msgr2=0x7f2cac0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 -- 172.21.15.64:0/1801497305 shutdown_connections 2024-04-01T05:48:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.363+0000 7f2cb32c9640 1 -- 172.21.15.64:0/1801497305 wait complete. 2024-04-01T05:48:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.366+0000 7f2cb32c9640 1 Processor -- start 2024-04-01T05:48:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.366+0000 7f2cb32c9640 1 -- start start 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cac0cbca0 0x7f2cac0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac06b7d0 con 0x7f2cac0cbca0 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0ceb70 con 0x7f2cac06ff70 2024-04-01T05:48:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb32c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac06b950 con 0x7f2cac070350 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb1065640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb1866640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cac0cbca0 0x7f2cac0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb0864640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb1065640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:33412/0 (socket says 172.21.15.64:33412) 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb1065640 1 -- 172.21.15.64:0/2719466998 learned_addr learned my addr 172.21.15.64:0/2719466998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.368+0000 7f2cb0864640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:48998/0 (socket says 172.21.15.64:48998) 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 -- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 msgr2=0x7f2cac0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 -- 172.21.15.64:0/2719466998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cac0cbca0 msgr2=0x7f2cac0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cac0cbca0 0x7f2cac0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 -- 172.21.15.64:0/2719466998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1f8ad0 con 0x7f2cac06ff70 2024-04-01T05:48:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2cb1065640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098c0 tx=0x7f2ca80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8032630 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.369+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca80327f0 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.370+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8032be0 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.370+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1f8db0 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.370+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1fb7f0 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.372+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f2ca804e430 con 0x7f2cac06ff70 2024-04-01T05:48:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.372+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac0688b0 con 0x7f2cac06ff70 2024-04-01T05:48:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.373+0000 7f2ca1ffb640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 0x7f2c9006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:48:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.373+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f2ca802f030 con 0x7f2cac06ff70 2024-04-01T05:48:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.373+0000 7f2cb0864640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 0x7f2c9006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:48:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.374+0000 7f2cb0864640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 0x7f2c9006de50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009560 tx=0x7f2c9c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:48:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.378+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f2ca8040070 con 0x7f2cac06ff70 2024-04-01T05:48:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.907+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f2cac0d0130 con 0x7f2c9006b990 2024-04-01T05:48:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.908+0000 7f2ca1ffb640 1 -- 172.21.15.64:0/2719466998 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2cac0d0130 con 0x7f2c9006b990 2024-04-01T05:48:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2024-04-01T05:48:59.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.911+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 msgr2=0x7f2c9006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.911+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 0x7f2c9006de50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009560 tx=0x7f2c9c009190 comp rx=0 tx=0).stop 2024-04-01T05:48:59.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.911+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 msgr2=0x7f2cac1fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.911+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098c0 tx=0x7f2ca80046c0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 shutdown_connections 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f2c9006b990 0x7f2c9006de50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cac0cbca0 0x7f2cac0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cac070350 0x7f2cac0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 --2- 172.21.15.64:0/2719466998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cac06ff70 0x7f2cac1fbff0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:48:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 >> 172.21.15.64:0/2719466998 conn(0x7f2cac0d0c70 msgr2=0x7f2cac06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:48:59.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 shutdown_connections 2024-04-01T05:48:59.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:48:59.912+0000 7f2cb32c9640 1 -- 172.21.15.64:0/2719466998 wait complete. 2024-04-01T05:48:59.936 INFO:tasks.ceph:Scrubbing osd.1 2024-04-01T05:48:59.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:00.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.226+0000 7f084e7d0640 1 Processor -- start 2024-04-01T05:49:00.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.226+0000 7f084e7d0640 1 -- start start 2024-04-01T05:49:00.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f084e7d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 0x7f0848072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f084e7d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 0x7f084815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f084e7d0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0848071ce0 con 0x7f08480cbb70 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f084e7d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f08480cbf50 con 0x7f084806b7d0 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f084e7d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f084815f310 con 0x7f084815e5e0 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f0847fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f08480cbb70 0x7f084806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48398/0 (socket says 172.21.15.64:48398) 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.228+0000 7f0847fff640 1 -- 172.21.15.64:0/1846422405 learned_addr learned my addr 172.21.15.64:0/1846422405 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:00.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f084cd6d640 1 --2- 172.21.15.64:0/1846422405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 0x7f084815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f08477fe640 1 --2- 172.21.15.64:0/1846422405 >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 0x7f0848072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f0846ffd640 1 -- 172.21.15.64:0/1846422405 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634141379 0 0) 0x7f0848071ce0 con 0x7f08480cbb70 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f0846ffd640 1 -- 172.21.15.64:0/1846422405 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0830003520 con 0x7f08480cbb70 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f084cd6d640 1 -- 172.21.15.64:0/1846422405 >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 msgr2=0x7f0848072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f084cd6d640 1 --2- 172.21.15.64:0/1846422405 >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 0x7f0848072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f084cd6d640 1 -- 172.21.15.64:0/1846422405 >> v1:172.21.15.64:6789/0 conn(0x7f08480cbb70 legacy=0x7f084806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f084cd6d640 1 -- 172.21.15.64:0/1846422405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0848071fa0 con 0x7f084815e5e0 2024-04-01T05:49:00.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.229+0000 7f08477fe640 1 --2- 172.21.15.64:0/1846422405 >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 0x7f0848072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:00.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.231+0000 7f084cd6d640 1 --2- 172.21.15.64:0/1846422405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 0x7f084815ea30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0840009d90 tx=0x7f084002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5da0cd9a5d6b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.232+0000 7f0846ffd640 1 -- 172.21.15.64:0/1846422405 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840039070 con 0x7f084815e5e0 2024-04-01T05:49:00.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.232+0000 7f0846ffd640 1 -- 172.21.15.64:0/1846422405 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084002cc50 con 0x7f084815e5e0 2024-04-01T05:49:00.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.232+0000 7f0846ffd640 1 -- 172.21.15.64:0/1846422405 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840034440 con 0x7f084815e5e0 2024-04-01T05:49:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 -- 172.21.15.64:0/1846422405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 msgr2=0x7f084815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 --2- 172.21.15.64:0/1846422405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 0x7f084815ea30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0840009d90 tx=0x7f084002bea0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 -- 172.21.15.64:0/1846422405 shutdown_connections 2024-04-01T05:49:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 --2- 172.21.15.64:0/1846422405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f084815e5e0 0x7f084815ea30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 --2- 172.21.15.64:0/1846422405 >> v2:172.21.15.100:3300/0 conn(0x7f084806b7d0 0x7f0848072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.233+0000 7f084e7d0640 1 -- 172.21.15.64:0/1846422405 >> 172.21.15.64:0/1846422405 conn(0x7f08480d0bd0 msgr2=0x7f08480d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.234+0000 7f084e7d0640 1 -- 172.21.15.64:0/1846422405 shutdown_connections 2024-04-01T05:49:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.234+0000 7f084e7d0640 1 -- 172.21.15.64:0/1846422405 wait complete. 2024-04-01T05:49:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.237+0000 7f084e7d0640 1 Processor -- start 2024-04-01T05:49:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.237+0000 7f084e7d0640 1 -- start start 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084806b7d0 0x7f0848156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08480cbb70 0x7f08481f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f084815e5e0 0x7f08481f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f084815ef70 con 0x7f084806b7d0 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0848072560 con 0x7f084815e5e0 2024-04-01T05:49:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f084e7d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f084815f0f0 con 0x7f08480cbb70 2024-04-01T05:49:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.239+0000 7f0847fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084806b7d0 0x7f0848156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f0847fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084806b7d0 0x7f0848156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44554/0 (socket says 172.21.15.64:44554) 2024-04-01T05:49:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f0847fff640 1 -- 172.21.15.64:0/2037781326 learned_addr learned my addr 172.21.15.64:0/2037781326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f084815e5e0 0x7f08481f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f08477fe640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08480cbb70 0x7f08481f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 -- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08480cbb70 msgr2=0x7f08481f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08480cbb70 0x7f08481f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 -- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084806b7d0 msgr2=0x7f0848156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084806b7d0 0x7f0848156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f084cd6d640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480b39f0 con 0x7f084815e5e0 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.240+0000 7f08477fe640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08480cbb70 0x7f08481f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f084cd6d640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f084815e5e0 0x7f08481f7740 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0840033000 tx=0x7f0840004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840039050 con 0x7f084815e5e0 2024-04-01T05:49:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0840045070 con 0x7f084815e5e0 2024-04-01T05:49:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08480b3cd0 con 0x7f084815e5e0 2024-04-01T05:49:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840009450 con 0x7f084815e5e0 2024-04-01T05:49:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.241+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08480b4290 con 0x7f084815e5e0 2024-04-01T05:49:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.243+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08480688b0 con 0x7f084815e5e0 2024-04-01T05:49:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.244+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f084003e610 con 0x7f084815e5e0 2024-04-01T05:49:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.245+0000 7f0844ff9640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f083406b990 0x7f083406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.245+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f08400acd60 con 0x7f084815e5e0 2024-04-01T05:49:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f0847fff640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f083406b990 0x7f083406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f0844ff9640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7f0834071850 0x7f0834073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f084002cc50 con 0x7f0834071850 2024-04-01T05:49:00.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f08400af380 con 0x7f084815e5e0 2024-04-01T05:49:00.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f08477fe640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7f0834071850 0x7f0834073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.246+0000 7f0847fff640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f083406b990 0x7f083406de50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0838005880 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.247+0000 7f08477fe640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7f0834071850 0x7f0834073c70 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.249+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== osd.1 v2:172.21.15.64:6816/199266641 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f084002cc50 con 0x7f0834071850 2024-04-01T05:49:00.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.288+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 --> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f08480d1990 con 0x7f0834071850 2024-04-01T05:49:00.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.288+0000 7f0844ff9640 1 -- 172.21.15.64:0/2037781326 <== osd.1 v2:172.21.15.64:6816/199266641 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f08480d1990 con 0x7f0834071850 2024-04-01T05:49:00.289 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:00.289 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7f0834071850 msgr2=0x7f0834073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.64:6816/199266641,v1:172.21.15.64:6817/199266641] conn(0x7f0834071850 0x7f0834073c70 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f083406b990 msgr2=0x7f083406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f083406b990 0x7f083406de50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0838005880 tx=0x7f0838009040 comp rx=0 tx=0).stop 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f084815e5e0 msgr2=0x7f08481f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 --2- 172.21.15.64:0/2037781326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f084815e5e0 0x7f08481f7740 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0840033000 tx=0x7f0840004a40 comp rx=0 tx=0).stop 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084cd6d640 1 -- 172.21.15.64:0/2037781326 reap_dead start 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 shutdown_connections 2024-04-01T05:49:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 >> 172.21.15.64:0/2037781326 conn(0x7f08480d0bd0 msgr2=0x7f08480ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 shutdown_connections 2024-04-01T05:49:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.289+0000 7f084e7d0640 1 -- 172.21.15.64:0/2037781326 wait complete. 2024-04-01T05:49:00.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-01T05:49:00.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.592+0000 7f11a9b56640 1 Processor -- start 2024-04-01T05:49:00.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.592+0000 7f11a9b56640 1 -- start start 2024-04-01T05:49:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.594+0000 7f11a9b56640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11a41589c0 0x7f11a40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.594+0000 7f11a9b56640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11a4071c30 0x7f11a415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.594+0000 7f11a9b56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a415b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a9b56640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f11a415f3e0 con 0x7f11a41589c0 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a9b56640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f11a4072480 con 0x7f11a4071c30 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a9b56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f11a4072760 con 0x7f11a415ef30 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11a41589c0 0x7f11a40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11a41589c0 0x7f11a40cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44556/0 (socket says 172.21.15.64:44556) 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 -- 172.21.15.64:0/2421831809 learned_addr learned my addr 172.21.15.64:0/2421831809 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f119affd640 1 --2- 172.21.15.64:0/2421831809 >> v2:172.21.15.100:3300/0 conn(0x7f11a4071c30 0x7f11a415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a3fff640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 -- 172.21.15.64:0/2421831809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 msgr2=0x7f11a415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:00.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a415b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 -- 172.21.15.64:0/2421831809 >> v2:172.21.15.100:3300/0 conn(0x7f11a4071c30 msgr2=0x7f11a415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:00.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.595+0000 7f11a37fe640 1 --2- 172.21.15.64:0/2421831809 >> v2:172.21.15.100:3300/0 conn(0x7f11a4071c30 0x7f11a415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.596+0000 7f11a37fe640 1 -- 172.21.15.64:0/2421831809 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a4072a40 con 0x7f11a41589c0 2024-04-01T05:49:00.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.598+0000 7f11a37fe640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40cfbb0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f11a4067ce0 tx=0x7f119402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4409be58a773d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.598+0000 7f11a2ffd640 1 -- 172.21.15.64:0/2421831809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035020 con 0x7f11a41589c0 2024-04-01T05:49:00.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.598+0000 7f11a2ffd640 1 -- 172.21.15.64:0/2421831809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1194031870 con 0x7f11a41589c0 2024-04-01T05:49:00.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.598+0000 7f11a2ffd640 1 -- 172.21.15.64:0/2421831809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194031c40 con 0x7f11a41589c0 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 -- 172.21.15.64:0/2421831809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 msgr2=0x7f11a40cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40cfbb0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f11a4067ce0 tx=0x7f119402b910 comp rx=0 tx=0).stop 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 -- 172.21.15.64:0/2421831809 shutdown_connections 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 --2- 172.21.15.64:0/2421831809 >> v2:172.21.15.100:3300/0 conn(0x7f11a4071c30 0x7f11a415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 --2- 172.21.15.64:0/2421831809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40cfbb0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.599+0000 7f11a9b56640 1 -- 172.21.15.64:0/2421831809 >> 172.21.15.64:0/2421831809 conn(0x7f11a40d0c40 msgr2=0x7f11a40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:00.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.600+0000 7f11a9b56640 1 -- 172.21.15.64:0/2421831809 shutdown_connections 2024-04-01T05:49:00.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.600+0000 7f11a9b56640 1 -- 172.21.15.64:0/2421831809 wait complete. 2024-04-01T05:49:00.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.603+0000 7f11a9b56640 1 Processor -- start 2024-04-01T05:49:00.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.603+0000 7f11a9b56640 1 -- start start 2024-04-01T05:49:00.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 0x7f11a40b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a4153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4158da0 con 0x7f11a41589c0 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40cbac0 con 0x7f11a4071c30 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a9b56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40720b0 con 0x7f11a415ef30 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f119affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f119affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44564/0 (socket says 172.21.15.64:44564) 2024-04-01T05:49:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f119affd640 1 -- 172.21.15.64:0/3834486565 learned_addr learned my addr 172.21.15.64:0/3834486565 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.605+0000 7f11a37fe640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 0x7f11a40b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a3fff640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a4153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a37fe640 1 -- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 msgr2=0x7f11a4153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a37fe640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a4153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a37fe640 1 -- 172.21.15.64:0/3834486565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 msgr2=0x7f11a40b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a37fe640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a37fe640 1 -- 172.21.15.64:0/3834486565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40c5240 con 0x7f11a4071c30 2024-04-01T05:49:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f119affd640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:00.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.606+0000 7f11a3fff640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a4153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:00.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a37fe640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 0x7f11a40b4240 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1194033000 tx=0x7f11940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194040070 con 0x7f11a4071c30 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1194031e50 con 0x7f11a4071c30 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41536b0 con 0x7f11a4071c30 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035040 con 0x7f11a4071c30 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.607+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a40b4c50 con 0x7f11a4071c30 2024-04-01T05:49:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.609+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a4068350 con 0x7f11a4071c30 2024-04-01T05:49:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.610+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f1194038020 con 0x7f11a4071c30 2024-04-01T05:49:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.610+0000 7f11a0ff9640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 0x7f1184070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.610+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f11940ac0c0 con 0x7f11a4071c30 2024-04-01T05:49:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.611+0000 7f119affd640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 0x7f1184070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.611+0000 7f119affd640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 0x7f1184070060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11a41f4a30 tx=0x7f118c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:00.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:00.616+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f11a4068350 con 0x7f11a4071c30 2024-04-01T05:49:01.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.150+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f11a40688b0 con 0x7f118406dba0 2024-04-01T05:49:01.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.151+0000 7f11a0ff9640 1 -- 172.21.15.64:0/3834486565 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f11a40688b0 con 0x7f118406dba0 2024-04-01T05:49:01.151 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2024-04-01T05:49:01.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.154+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 msgr2=0x7f1184070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.154+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 0x7f1184070060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11a41f4a30 tx=0x7f118c009190 comp rx=0 tx=0).stop 2024-04-01T05:49:01.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.154+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 msgr2=0x7f11a40b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.154+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 0x7f11a40b4240 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1194033000 tx=0x7f11940046c0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 shutdown_connections 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f118406dba0 0x7f1184070060 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11a415ef30 0x7f11a4153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a41589c0 0x7f11a40b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 --2- 172.21.15.64:0/3834486565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a4071c30 0x7f11a40b4240 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 >> 172.21.15.64:0/3834486565 conn(0x7f11a40d0c40 msgr2=0x7f11a40cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 shutdown_connections 2024-04-01T05:49:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.155+0000 7f11a9b56640 1 -- 172.21.15.64:0/3834486565 wait complete. 2024-04-01T05:49:01.180 INFO:tasks.ceph:Scrubbing osd.2 2024-04-01T05:49:01.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.469+0000 7f3a0cacd640 1 Processor -- start 2024-04-01T05:49:01.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.469+0000 7f3a0cacd640 1 -- start start 2024-04-01T05:49:01.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a0cacd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a081564d0 0x7f3a08160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a0cacd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a08160980 con 0x7f3a0815c440 2024-04-01T05:49:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a0cacd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a0815cf70 con 0x7f3a0806b7d0 2024-04-01T05:49:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a0cacd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3a0815d250 con 0x7f3a081564d0 2024-04-01T05:49:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a05d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3a0815c440 0x7f3a0806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48412/0 (socket says 172.21.15.64:48412) 2024-04-01T05:49:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a05d9b640 1 -- 172.21.15.64:0/3688338496 learned_addr learned my addr 172.21.15.64:0/3688338496 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.472+0000 7f3a0659c640 1 --2- 172.21.15.64:0/3688338496 >> v2:172.21.15.100:3300/0 conn(0x7f3a081564d0 0x7f3a08160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3570640296 0 0) 0x7f3a0815cf70 con 0x7f3a0806b7d0 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f4003520 con 0x7f3a0806b7d0 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150309003 0 0) 0x7f3a08160980 con 0x7f3a0815c440 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a0815cf70 con 0x7f3a0815c440 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0659c640 1 -- 172.21.15.64:0/3688338496 >> v1:172.21.15.100:6789/0 conn(0x7f3a0806b7d0 legacy=0x7f3a080b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0659c640 1 -- 172.21.15.64:0/3688338496 >> v1:172.21.15.64:6789/0 conn(0x7f3a0815c440 legacy=0x7f3a0806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.473+0000 7f3a0659c640 1 -- 172.21.15.64:0/3688338496 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0815fc40 con 0x7f3a081564d0 2024-04-01T05:49:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.475+0000 7f3a0659c640 1 --2- 172.21.15.64:0/3688338496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a08160440 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f39f00097a0 tx=0x7f39f002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1074363e3997047 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.475+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0034070 con 0x7f3a081564d0 2024-04-01T05:49:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.476+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f0030610 con 0x7f3a081564d0 2024-04-01T05:49:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.476+0000 7f3a0559a640 1 -- 172.21.15.64:0/3688338496 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f00309e0 con 0x7f3a081564d0 2024-04-01T05:49:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 -- 172.21.15.64:0/3688338496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 msgr2=0x7f3a08160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 --2- 172.21.15.64:0/3688338496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a08160440 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f39f00097a0 tx=0x7f39f002b960 comp rx=0 tx=0).stop 2024-04-01T05:49:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 -- 172.21.15.64:0/3688338496 shutdown_connections 2024-04-01T05:49:01.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 --2- 172.21.15.64:0/3688338496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a08160440 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 -- 172.21.15.64:0/3688338496 >> 172.21.15.64:0/3688338496 conn(0x7f3a0806c1d0 msgr2=0x7f3a0806c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:01.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 -- 172.21.15.64:0/3688338496 shutdown_connections 2024-04-01T05:49:01.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.477+0000 7f3a0cacd640 1 -- 172.21.15.64:0/3688338496 wait complete. 2024-04-01T05:49:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.480+0000 7f3a0cacd640 1 Processor -- start 2024-04-01T05:49:01.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.481+0000 7f3a0cacd640 1 -- start start 2024-04-01T05:49:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 0x7f3a0806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a080b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a0815c440 0x7f3a081541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080adef0 con 0x7f3a0815c440 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081568b0 con 0x7f3a081564d0 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0cacd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0815c8c0 con 0x7f3a0806b7d0 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 0x7f3a0806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a05d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a080b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a06d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a0815c440 0x7f3a081541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 0x7f3a0806e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:49058/0 (socket says 172.21.15.64:49058) 2024-04-01T05:49:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.483+0000 7f3a0659c640 1 -- 172.21.15.64:0/1169818355 learned_addr learned my addr 172.21.15.64:0/1169818355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:01.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 -- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 msgr2=0x7f3a080b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a080b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 -- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a0815c440 msgr2=0x7f3a081541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a0815c440 0x7f3a081541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a080b0a70 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a05d9b640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a081564d0 0x7f3a080b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.484+0000 7f3a0659c640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 0x7f3a0806e6e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39f00098c0 tx=0x7f39f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.485+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0034040 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.485+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f0030410 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.485+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0006e10 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.485+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0806ed70 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.485+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0806f330 con 0x7f3a0806b7d0 2024-04-01T05:49:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.487+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a080688b0 con 0x7f3a0806b7d0 2024-04-01T05:49:01.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.488+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f39f0037070 con 0x7f3a0806b7d0 2024-04-01T05:49:01.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.489+0000 7f39feffd640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39e806b990 0x7f39e806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.489+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f39f00ac2c0 con 0x7f3a0806b7d0 2024-04-01T05:49:01.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.489+0000 7f39feffd640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f39e8071850 0x7f39e8073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.489+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39e8074320 con 0x7f39e8071850 2024-04-01T05:49:01.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.489+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f39f00b3050 con 0x7f3a0806b7d0 2024-04-01T05:49:01.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.490+0000 7f3a05d9b640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39e806b990 0x7f39e806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.490+0000 7f3a06d9d640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f39e8071850 0x7f39e8073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.490+0000 7f3a05d9b640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39e806b990 0x7f39e806de50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3a080cf920 tx=0x7f39f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.491+0000 7f3a06d9d640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f39e8071850 0x7f39e8073c70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.492+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== osd.2 v2:172.21.15.64:6824/2000381811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39e8074320 con 0x7f39e8071850 2024-04-01T05:49:01.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.531+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 --> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3a0815b860 con 0x7f39e8071850 2024-04-01T05:49:01.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.531+0000 7f39feffd640 1 -- 172.21.15.64:0/1169818355 <== osd.2 v2:172.21.15.64:6824/2000381811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3a0815b860 con 0x7f39e8071850 2024-04-01T05:49:01.532 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:01.532 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:01.533 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:01.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f39e8071850 msgr2=0x7f39e8073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.64:6824/2000381811,v1:172.21.15.64:6825/2000381811] conn(0x7f39e8071850 0x7f39e8073c70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39e806b990 msgr2=0x7f39e806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f39e806b990 0x7f39e806de50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3a080cf920 tx=0x7f39f8009040 comp rx=0 tx=0).stop 2024-04-01T05:49:01.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 msgr2=0x7f3a0806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a0cacd640 1 --2- 172.21.15.64:0/1169818355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a0806b7d0 0x7f3a0806e6e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39f00098c0 tx=0x7f39f00046c0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.532+0000 7f3a06d9d640 1 -- 172.21.15.64:0/1169818355 reap_dead start 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.533+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 shutdown_connections 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.533+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 >> 172.21.15.64:0/1169818355 conn(0x7f3a0806c1d0 msgr2=0x7f3a0815d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.533+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 shutdown_connections 2024-04-01T05:49:01.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.533+0000 7f3a0cacd640 1 -- 172.21.15.64:0/1169818355 wait complete. 2024-04-01T05:49:01.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-01T05:49:01.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.833+0000 7fd1b009d640 1 Processor -- start 2024-04-01T05:49:01.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.833+0000 7fd1b009d640 1 -- start start 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.835+0000 7fd1b009d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1a815bef0 0x7fd1a81603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.835+0000 7fd1b009d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 0x7fd1a806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1b009d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1b009d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a8160930 con 0x7fd1a806b600 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1b009d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a815c2d0 con 0x7fd1a815bef0 2024-04-01T05:49:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1b009d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a80d0650 con 0x7fd1a8071c30 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ae63a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ae63a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44584/0 (socket says 172.21.15.64:44584) 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ae63a640 1 -- 172.21.15.64:0/2025563295 learned_addr learned my addr 172.21.15.64:0/2025563295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2025563295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 0x7fd1a806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ade39640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.100:3300/0 conn(0x7fd1a815bef0 0x7fd1a81603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ad638640 1 -- 172.21.15.64:0/2025563295 >> v2:172.21.15.100:3300/0 conn(0x7fd1a815bef0 msgr2=0x7fd1a81603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.836+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.100:3300/0 conn(0x7fd1a815bef0 0x7fd1a81603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.837+0000 7fd1ad638640 1 -- 172.21.15.64:0/2025563295 >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 msgr2=0x7fd1a815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:01.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.837+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.837+0000 7fd1ad638640 1 -- 172.21.15.64:0/2025563295 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a80d0930 con 0x7fd1a8071c30 2024-04-01T05:49:01.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.837+0000 7fd1ae63a640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:01.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.839+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2025563295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 0x7fd1a806b0c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd1a806fbe0 tx=0x7fd19802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8ad5510b4ceb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.839+0000 7fd1ace37640 1 -- 172.21.15.64:0/2025563295 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198039070 con 0x7fd1a8071c30 2024-04-01T05:49:01.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.839+0000 7fd1ace37640 1 -- 172.21.15.64:0/2025563295 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1980316f0 con 0x7fd1a8071c30 2024-04-01T05:49:01.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.839+0000 7fd1ace37640 1 -- 172.21.15.64:0/2025563295 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198031ac0 con 0x7fd1a8071c30 2024-04-01T05:49:01.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 -- 172.21.15.64:0/2025563295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 msgr2=0x7fd1a806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2025563295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 0x7fd1a806b0c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd1a806fbe0 tx=0x7fd19802b790 comp rx=0 tx=0).stop 2024-04-01T05:49:01.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 -- 172.21.15.64:0/2025563295 shutdown_connections 2024-04-01T05:49:01.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.64:3300/0 conn(0x7fd1a806b600 0x7fd1a815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2025563295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a8071c30 0x7fd1a806b0c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2025563295 >> v2:172.21.15.100:3300/0 conn(0x7fd1a815bef0 0x7fd1a81603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.840+0000 7fd1b009d640 1 -- 172.21.15.64:0/2025563295 >> 172.21.15.64:0/2025563295 conn(0x7fd1a80d0c40 msgr2=0x7fd1a80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:01.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.841+0000 7fd1b009d640 1 -- 172.21.15.64:0/2025563295 shutdown_connections 2024-04-01T05:49:01.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.841+0000 7fd1b009d640 1 -- 172.21.15.64:0/2025563295 wait complete. 2024-04-01T05:49:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.844+0000 7fd1b009d640 1 Processor -- start 2024-04-01T05:49:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.844+0000 7fd1b009d640 1 -- start start 2024-04-01T05:49:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a8071c30 0x7fd1a814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a815bef0 0x7fd1a80c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a80d0340 con 0x7fd1a806b600 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a80cd2c0 con 0x7fd1a8071c30 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1b009d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a80ccc20 con 0x7fd1a815bef0 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1ade39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1ade39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44600/0 (socket says 172.21.15.64:44600) 2024-04-01T05:49:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1ade39640 1 -- 172.21.15.64:0/2935355737 learned_addr learned my addr 172.21.15.64:0/2935355737 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a8071c30 0x7fd1a814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.846+0000 7fd1ae63a640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a815bef0 0x7fd1a80c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 -- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a815bef0 msgr2=0x7fd1a80c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a815bef0 0x7fd1a80c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 -- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a8071c30 msgr2=0x7fd1a814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a8071c30 0x7fd1a814d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 -- 172.21.15.64:0/2935355737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a80c6c60 con 0x7fd1a806b600 2024-04-01T05:49:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1ade39640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e870 tx=0x7fd1a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4017070 con 0x7fd1a806b600 2024-04-01T05:49:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a400c8e0 con 0x7fd1a806b600 2024-04-01T05:49:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a80c83d0 con 0x7fd1a806b600 2024-04-01T05:49:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400ccd0 con 0x7fd1a806b600 2024-04-01T05:49:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.847+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a80c8990 con 0x7fd1a806b600 2024-04-01T05:49:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.850+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a80688b0 con 0x7fd1a806b600 2024-04-01T05:49:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.850+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd1a401a070 con 0x7fd1a806b600 2024-04-01T05:49:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.851+0000 7fd19e7fc640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 0x7fd18c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:01.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.851+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd1a408a340 con 0x7fd1a806b600 2024-04-01T05:49:01.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.852+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 0x7fd18c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:01.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.853+0000 7fd1ad638640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 0x7fd18c06de50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd198036000 tx=0x7fd198008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:01.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:01.864+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd1a4058c00 con 0x7fd1a806b600 2024-04-01T05:49:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.415+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fd1a802d150 con 0x7fd18c06b990 2024-04-01T05:49:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.416+0000 7fd19e7fc640 1 -- 172.21.15.64:0/2935355737 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd1a802d150 con 0x7fd18c06b990 2024-04-01T05:49:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2024-04-01T05:49:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 msgr2=0x7fd18c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 0x7fd18c06de50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd198036000 tx=0x7fd198008030 comp rx=0 tx=0).stop 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 msgr2=0x7fd1a80c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e870 tx=0x7fd1a400ed30 comp rx=0 tx=0).stop 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 shutdown_connections 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd18c06b990 0x7fd18c06de50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a815bef0 0x7fd1a80c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a8071c30 0x7fd1a814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 --2- 172.21.15.64:0/2935355737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806b600 0x7fd1a80c7d10 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 >> 172.21.15.64:0/2935355737 conn(0x7fd1a80d0c40 msgr2=0x7fd1a80b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 shutdown_connections 2024-04-01T05:49:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.420+0000 7fd1b009d640 1 -- 172.21.15.64:0/2935355737 wait complete. 2024-04-01T05:49:02.445 INFO:tasks.ceph:Scrubbing osd.3 2024-04-01T05:49:02.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:02.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.734+0000 7fd217702640 1 Processor -- start 2024-04-01T05:49:02.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.734+0000 7fd217702640 1 -- start start 2024-04-01T05:49:02.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd217702640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd210166ba0 0x7fd210168f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd217702640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100727c0 con 0x7fd21006f6f0 2024-04-01T05:49:02.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd217702640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd210072aa0 con 0x7fd21006b0c0 2024-04-01T05:49:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd217702640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd21006e250 con 0x7fd210166ba0 2024-04-01T05:49:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd214c9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd21006f6f0 0x7fd210071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48426/0 (socket says 172.21.15.64:48426) 2024-04-01T05:49:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd214c9d640 1 -- 172.21.15.64:0/2262531938 learned_addr learned my addr 172.21.15.64:0/2262531938 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd215c9f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd210166ba0 0x7fd210168f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.737+0000 7fd21549e640 1 --1- 172.21.15.64:0/2262531938 >> v1:172.21.15.100:6789/0 conn(0x7fd21006b0c0 0x7fd21006b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:35632/0 (socket says 172.21.15.64:35632) 2024-04-01T05:49:02.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817302025 0 0) 0x7fd2100727c0 con 0x7fd21006f6f0 2024-04-01T05:49:02.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1fc003520 con 0x7fd21006f6f0 2024-04-01T05:49:02.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858298691 0 0) 0x7fd210072aa0 con 0x7fd21006b0c0 2024-04-01T05:49:02.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2100727c0 con 0x7fd21006b0c0 2024-04-01T05:49:02.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd215c9f640 1 -- 172.21.15.64:0/2262531938 >> v1:172.21.15.100:6789/0 conn(0x7fd21006b0c0 legacy=0x7fd21006b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd215c9f640 1 -- 172.21.15.64:0/2262531938 >> v1:172.21.15.64:6789/0 conn(0x7fd21006f6f0 legacy=0x7fd210071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.738+0000 7fd215c9f640 1 -- 172.21.15.64:0/2262531938 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21006dee0 con 0x7fd210166ba0 2024-04-01T05:49:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.740+0000 7fd215c9f640 1 --2- 172.21.15.64:0/2262531938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210166ba0 0x7fd210168f90 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad175b88fca200d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.740+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208039070 con 0x7fd210166ba0 2024-04-01T05:49:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.740+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20802cc60 con 0x7fd210166ba0 2024-04-01T05:49:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.741+0000 7fd207fff640 1 -- 172.21.15.64:0/2262531938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208034660 con 0x7fd210166ba0 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 -- 172.21.15.64:0/2262531938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210166ba0 msgr2=0x7fd210168f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 --2- 172.21.15.64:0/2262531938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210166ba0 0x7fd210168f90 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20802beb0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 -- 172.21.15.64:0/2262531938 shutdown_connections 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 --2- 172.21.15.64:0/2262531938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210166ba0 0x7fd210168f90 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 -- 172.21.15.64:0/2262531938 >> 172.21.15.64:0/2262531938 conn(0x7fd2100b2760 msgr2=0x7fd2100b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 -- 172.21.15.64:0/2262531938 shutdown_connections 2024-04-01T05:49:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.742+0000 7fd217702640 1 -- 172.21.15.64:0/2262531938 wait complete. 2024-04-01T05:49:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.745+0000 7fd217702640 1 Processor -- start 2024-04-01T05:49:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.745+0000 7fd217702640 1 -- start start 2024-04-01T05:49:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.747+0000 7fd217702640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 0x7fd210157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.747+0000 7fd217702640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd21006f6f0 0x7fd21015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd217702640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd210166ba0 0x7fd21013e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd217702640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd210147b40 con 0x7fd21006b0c0 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd217702640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd210072430 con 0x7fd21006f6f0 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd217702640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd210147cc0 con 0x7fd210166ba0 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd21549e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 0x7fd210157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd21549e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 0x7fd210157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44622/0 (socket says 172.21.15.64:44622) 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd21549e640 1 -- 172.21.15.64:0/977987702 learned_addr learned my addr 172.21.15.64:0/977987702 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd214c9d640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd21006f6f0 0x7fd21015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd215c9f640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd210166ba0 0x7fd21013e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.748+0000 7fd215c9f640 1 -- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd21006f6f0 msgr2=0x7fd21015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd215c9f640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd21006f6f0 0x7fd21015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd215c9f640 1 -- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 msgr2=0x7fd210157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd215c9f640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 0x7fd210157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd215c9f640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21015ad70 con 0x7fd210166ba0 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd214c9d640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd21006f6f0 0x7fd21015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd21549e640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21006b0c0 0x7fd210157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd215c9f640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd210166ba0 0x7fd21013e0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd208033000 tx=0x7fd208004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208039050 con 0x7fd210166ba0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208045070 con 0x7fd210166ba0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.749+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21015b010 con 0x7fd210166ba0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.750+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208009450 con 0x7fd210166ba0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.750+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21015b5d0 con 0x7fd210166ba0 2024-04-01T05:49:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.752+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2100688b0 con 0x7fd210166ba0 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.753+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd20803e850 con 0x7fd210166ba0 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd205ffb640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd1f406b990 0x7fd1f406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd2080ad090 con 0x7fd210166ba0 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd21549e640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd1f406b990 0x7fd1f406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd205ffb640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7fd1f4071850 0x7fd1f4073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd20802cc60 con 0x7fd1f4071850 2024-04-01T05:49:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd2080b4050 con 0x7fd210166ba0 2024-04-01T05:49:02.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.754+0000 7fd214c9d640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7fd1f4071850 0x7fd1f4073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:02.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.755+0000 7fd21549e640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd1f406b990 0x7fd1f406de50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0011e0 tx=0x7fd20c007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:02.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.755+0000 7fd214c9d640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7fd1f4071850 0x7fd1f4073c70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:02.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.757+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== osd.3 v2:172.21.15.64:6808/2132154805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd20802cc60 con 0x7fd1f4071850 2024-04-01T05:49:02.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.795+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 --> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd2100b49d0 con 0x7fd1f4071850 2024-04-01T05:49:02.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.796+0000 7fd205ffb640 1 -- 172.21.15.64:0/977987702 <== osd.3 v2:172.21.15.64:6808/2132154805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd2100b49d0 con 0x7fd1f4071850 2024-04-01T05:49:02.796 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:02.797 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7fd1f4071850 msgr2=0x7fd1f4073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.64:6808/2132154805,v1:172.21.15.64:6809/2132154805] conn(0x7fd1f4071850 0x7fd1f4073c70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd1f406b990 msgr2=0x7fd1f406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd1f406b990 0x7fd1f406de50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0011e0 tx=0x7fd20c007df0 comp rx=0 tx=0).stop 2024-04-01T05:49:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd210166ba0 msgr2=0x7fd21013e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 --2- 172.21.15.64:0/977987702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd210166ba0 0x7fd21013e0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd208033000 tx=0x7fd208004a40 comp rx=0 tx=0).stop 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd215c9f640 1 -- 172.21.15.64:0/977987702 reap_dead start 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 shutdown_connections 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 >> 172.21.15.64:0/977987702 conn(0x7fd2100b2760 msgr2=0x7fd210141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 shutdown_connections 2024-04-01T05:49:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:02.797+0000 7fd217702640 1 -- 172.21.15.64:0/977987702 wait complete. 2024-04-01T05:49:02.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-01T05:49:03.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.098+0000 7f8282f3e640 1 Processor -- start 2024-04-01T05:49:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.098+0000 7f8282f3e640 1 -- start start 2024-04-01T05:49:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f827c1589c0 0x7f827c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 0x7f827c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f827c15f3e0 con 0x7f827c1589c0 2024-04-01T05:49:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f827c072480 con 0x7f827c071c30 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.100+0000 7f8282f3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f827c072760 con 0x7f827c15ef30 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f827c1589c0 0x7f827c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f827c1589c0 0x7f827c0cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44624/0 (socket says 172.21.15.64:44624) 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 -- 172.21.15.64:0/756360959 learned_addr learned my addr 172.21.15.64:0/756360959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8273fff640 1 --2- 172.21.15.64:0/756360959 >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 0x7f827c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f82814db640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 -- 172.21.15.64:0/756360959 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 msgr2=0x7f827c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 -- 172.21.15.64:0/756360959 >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 msgr2=0x7f827c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 --2- 172.21.15.64:0/756360959 >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 0x7f827c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.101+0000 7f8280cda640 1 -- 172.21.15.64:0/756360959 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c072a40 con 0x7f827c1589c0 2024-04-01T05:49:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.102+0000 7f82814db640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.102+0000 7f8273fff640 1 --2- 172.21.15.64:0/756360959 >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 0x7f827c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.103+0000 7f8280cda640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c1589c0 0x7f827c0cfbb0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f827c067ce0 tx=0x7f827802e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b9dbf185467d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.104+0000 7f82737fe640 1 -- 172.21.15.64:0/756360959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827803b070 con 0x7f827c1589c0 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.104+0000 7f82737fe640 1 -- 172.21.15.64:0/756360959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8278036440 con 0x7f827c1589c0 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.104+0000 7f82737fe640 1 -- 172.21.15.64:0/756360959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278036810 con 0x7f827c1589c0 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 -- 172.21.15.64:0/756360959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c1589c0 msgr2=0x7f827c0cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c1589c0 0x7f827c0cfbb0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f827c067ce0 tx=0x7f827802e0d0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 -- 172.21.15.64:0/756360959 shutdown_connections 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 --2- 172.21.15.64:0/756360959 >> v2:172.21.15.100:3300/0 conn(0x7f827c071c30 0x7f827c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 --2- 172.21.15.64:0/756360959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c1589c0 0x7f827c0cfbb0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 -- 172.21.15.64:0/756360959 >> 172.21.15.64:0/756360959 conn(0x7f827c0d0c40 msgr2=0x7f827c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.105+0000 7f8282f3e640 1 -- 172.21.15.64:0/756360959 shutdown_connections 2024-04-01T05:49:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.106+0000 7f8282f3e640 1 -- 172.21.15.64:0/756360959 wait complete. 2024-04-01T05:49:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.109+0000 7f8282f3e640 1 Processor -- start 2024-04-01T05:49:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.109+0000 7f8282f3e640 1 -- start start 2024-04-01T05:49:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 0x7f827c0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f827c158da0 con 0x7f827c071c30 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f827c0cbac0 con 0x7f827c1589c0 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8282f3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f827c0720b0 con 0x7f827c15ef30 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8280cda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f82814db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8280cda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44634/0 (socket says 172.21.15.64:44634) 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8280cda640 1 -- 172.21.15.64:0/736399439 learned_addr learned my addr 172.21.15.64:0/736399439 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.111+0000 7f8273fff640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 0x7f827c0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 -- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 msgr2=0x7f827c0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c0b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 -- 172.21.15.64:0/736399439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 msgr2=0x7f827c1efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 -- 172.21.15.64:0/736399439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c15a710 con 0x7f827c1589c0 2024-04-01T05:49:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f82814db640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c0b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8280cda640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.112+0000 7f8273fff640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 0x7f827c0b3830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f827c0d00f0 tx=0x7f826c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.113+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c017020 con 0x7f827c1589c0 2024-04-01T05:49:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.113+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c00ee60 con 0x7f827c1589c0 2024-04-01T05:49:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.113+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c15a9f0 con 0x7f827c1589c0 2024-04-01T05:49:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.113+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004480 con 0x7f827c1589c0 2024-04-01T05:49:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.113+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c154d10 con 0x7f827c1589c0 2024-04-01T05:49:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.116+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c068350 con 0x7f827c1589c0 2024-04-01T05:49:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.116+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f826c01a020 con 0x7f827c1589c0 2024-04-01T05:49:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.117+0000 7f82717fa640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 0x7f826806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.117+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f826c08a280 con 0x7f827c1589c0 2024-04-01T05:49:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.117+0000 7f8280cda640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 0x7f826806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.118+0000 7f8280cda640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 0x7f826806de50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8278039040 tx=0x7f8278032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.122+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f827c068350 con 0x7f827c1589c0 2024-04-01T05:49:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.650+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f827c0d1770 con 0x7f826806b990 2024-04-01T05:49:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.650+0000 7f82717fa640 1 -- 172.21.15.64:0/736399439 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f827c0d1770 con 0x7f826806b990 2024-04-01T05:49:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2024-04-01T05:49:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 msgr2=0x7f826806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 0x7f826806de50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8278039040 tx=0x7f8278032000 comp rx=0 tx=0).stop 2024-04-01T05:49:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 msgr2=0x7f827c0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 0x7f827c0b3830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f827c0d00f0 tx=0x7f826c00ec10 comp rx=0 tx=0).stop 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 shutdown_connections 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f826806b990 0x7f826806de50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f827c15ef30 0x7f827c0b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.655+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c1589c0 0x7f827c0b3830 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.656+0000 7f8282f3e640 1 --2- 172.21.15.64:0/736399439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f827c071c30 0x7f827c1efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.656+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 >> 172.21.15.64:0/736399439 conn(0x7f827c0d0c40 msgr2=0x7f827c1523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.656+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 shutdown_connections 2024-04-01T05:49:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.656+0000 7f8282f3e640 1 -- 172.21.15.64:0/736399439 wait complete. 2024-04-01T05:49:03.680 INFO:tasks.ceph:Scrubbing osd.4 2024-04-01T05:49:03.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:03.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.971+0000 7efd5b5c9640 1 Processor -- start 2024-04-01T05:49:03.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.971+0000 7efd5b5c9640 1 -- start start 2024-04-01T05:49:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.973+0000 7efd5b5c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd540cba90 0x7efd5406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd5b5c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd54071cc0 con 0x7efd5406b600 2024-04-01T05:49:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd5b5c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd5415f400 con 0x7efd5415ede0 2024-04-01T05:49:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd5b5c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efd54071f80 con 0x7efd540cba90 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd540cba90 0x7efd5406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd540cba90 0x7efd5406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:52370/0 (socket says 172.21.15.64:52370) 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 -- 172.21.15.64:0/2611578488 learned_addr learned my addr 172.21.15.64:0/2611578488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd58b64640 1 --1- 172.21.15.64:0/2611578488 >> v1:172.21.15.64:6789/0 conn(0x7efd5406b600 0x7efd5415e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33840/0 (socket says 172.21.15.64:33840) 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 -- 172.21.15.64:0/2611578488 >> v1:172.21.15.100:6789/0 conn(0x7efd5415ede0 legacy=0x7efd54167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 -- 172.21.15.64:0/2611578488 >> v1:172.21.15.64:6789/0 conn(0x7efd5406b600 legacy=0x7efd5415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.974+0000 7efd59365640 1 -- 172.21.15.64:0/2611578488 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd54072260 con 0x7efd540cba90 2024-04-01T05:49:03.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.976+0000 7efd59365640 1 --2- 172.21.15.64:0/2611578488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd540cba90 0x7efd5406b0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7efd50009870 tx=0x7efd5002ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1654b2abcbebcde server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.976+0000 7efd4bfff640 1 -- 172.21.15.64:0/2611578488 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5002f850 con 0x7efd540cba90 2024-04-01T05:49:03.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.976+0000 7efd4bfff640 1 -- 172.21.15.64:0/2611578488 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5002f9f0 con 0x7efd540cba90 2024-04-01T05:49:03.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.976+0000 7efd4bfff640 1 -- 172.21.15.64:0/2611578488 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5002fdc0 con 0x7efd540cba90 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 -- 172.21.15.64:0/2611578488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd540cba90 msgr2=0x7efd5406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 --2- 172.21.15.64:0/2611578488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd540cba90 0x7efd5406b0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7efd50009870 tx=0x7efd5002ba30 comp rx=0 tx=0).stop 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 -- 172.21.15.64:0/2611578488 shutdown_connections 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 --2- 172.21.15.64:0/2611578488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd540cba90 0x7efd5406b0c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 -- 172.21.15.64:0/2611578488 >> 172.21.15.64:0/2611578488 conn(0x7efd540d0bc0 msgr2=0x7efd540d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.977+0000 7efd5b5c9640 1 -- 172.21.15.64:0/2611578488 shutdown_connections 2024-04-01T05:49:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.978+0000 7efd5b5c9640 1 -- 172.21.15.64:0/2611578488 wait complete. 2024-04-01T05:49:03.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.980+0000 7efd5b5c9640 1 Processor -- start 2024-04-01T05:49:03.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.981+0000 7efd5b5c9640 1 -- start start 2024-04-01T05:49:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.982+0000 7efd5b5c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 0x7efd5406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.982+0000 7efd5b5c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd540cba90 0x7efd540b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd5b5c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5415ede0 0x7efd541571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd5b5c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd54072a60 con 0x7efd5415ede0 2024-04-01T05:49:03.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd5b5c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd540cbe70 con 0x7efd5406b600 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd5b5c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efd540c9360 con 0x7efd540cba90 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 0x7efd5406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 0x7efd5406ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:52384/0 (socket says 172.21.15.64:52384) 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 -- 172.21.15.64:0/1183691216 learned_addr learned my addr 172.21.15.64:0/1183691216 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59b66640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5415ede0 0x7efd541571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd58b64640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd540cba90 0x7efd540b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 -- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd540cba90 msgr2=0x7efd540b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd540cba90 0x7efd540b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 -- 172.21.15.64:0/1183691216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5415ede0 msgr2=0x7efd541571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5415ede0 0x7efd541571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd59365640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd540c9600 con 0x7efd5406b600 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.983+0000 7efd58b64640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd540cba90 0x7efd540b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd59365640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 0x7efd5406ef80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd50004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5002f840 con 0x7efd5406b600 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd50046070 con 0x7efd5406b600 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd540c98e0 con 0x7efd5406b600 2024-04-01T05:49:03.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50041620 con 0x7efd5406b600 2024-04-01T05:49:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.984+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5406f4c0 con 0x7efd5406b600 2024-04-01T05:49:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.986+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7efd50040070 con 0x7efd5406b600 2024-04-01T05:49:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.987+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd540688b0 con 0x7efd5406b600 2024-04-01T05:49:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.987+0000 7efd49ffb640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efd3806bb30 0x7efd3806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.987+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7efd500ace90 con 0x7efd5406b600 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd58b64640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efd3806bb30 0x7efd3806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd49ffb640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7efd380719f0 0x7efd38073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd50008f30 con 0x7efd380719f0 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd500af4b0 con 0x7efd5406b600 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd59b66640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7efd380719f0 0x7efd38073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.988+0000 7efd58b64640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efd3806bb30 0x7efd3806dff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efd44007fd0 tx=0x7efd44007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.989+0000 7efd59b66640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7efd380719f0 0x7efd38073e10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:03.990+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== osd.4 v2:172.21.15.100:6816/3826994341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd50008f30 con 0x7efd380719f0 2024-04-01T05:49:04.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.028+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 --> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd540d19c0 con 0x7efd380719f0 2024-04-01T05:49:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.029+0000 7efd49ffb640 1 -- 172.21.15.64:0/1183691216 <== osd.4 v2:172.21.15.100:6816/3826994341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd540d19c0 con 0x7efd380719f0 2024-04-01T05:49:04.029 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:04.029 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.029+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7efd380719f0 msgr2=0x7efd38073e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.029+0000 7efd5b5c9640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6816/3826994341,v1:172.21.15.100:6817/3826994341] conn(0x7efd380719f0 0x7efd38073e10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efd3806bb30 msgr2=0x7efd3806dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efd3806bb30 0x7efd3806dff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efd44007fd0 tx=0x7efd44007d60 comp rx=0 tx=0).stop 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 msgr2=0x7efd5406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 --2- 172.21.15.64:0/1183691216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5406b600 0x7efd5406ef80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd50004730 comp rx=0 tx=0).stop 2024-04-01T05:49:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd59b66640 1 -- 172.21.15.64:0/1183691216 reap_dead start 2024-04-01T05:49:04.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 shutdown_connections 2024-04-01T05:49:04.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 >> 172.21.15.64:0/1183691216 conn(0x7efd540d0bc0 msgr2=0x7efd541601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:04.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 shutdown_connections 2024-04-01T05:49:04.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.030+0000 7efd5b5c9640 1 -- 172.21.15.64:0/1183691216 wait complete. 2024-04-01T05:49:04.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-01T05:49:04.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.333+0000 7f0a76e23640 1 Processor -- start 2024-04-01T05:49:04.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.333+0000 7f0a76e23640 1 -- start start 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.335+0000 7f0a76e23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 0x7f0a700ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.335+0000 7f0a76e23640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 0x7f0a700cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.335+0000 7f0a76e23640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a700cd440 con 0x7f0a700ccca0 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.335+0000 7f0a76e23640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f0a700cf590 con 0x7f0a700cebf0 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.335+0000 7f0a76e23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0a700cf050 con 0x7f0a7015c060 2024-04-01T05:49:04.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a6ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 0x7f0a700cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a753c0640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0a700ccca0 0x7f0a700d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:55558/0 (socket says 172.21.15.64:55558) 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a753c0640 1 -- 172.21.15.64:0/4039710263 learned_addr learned my addr 172.21.15.64:0/4039710263 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/4039710263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 0x7f0a700ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a6f7fe640 1 -- 172.21.15.64:0/4039710263 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971480947 0 0) 0x7f0a700cd440 con 0x7f0a700ccca0 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a6f7fe640 1 -- 172.21.15.64:0/4039710263 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a58003520 con 0x7f0a700ccca0 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a74bbf640 1 -- 172.21.15.64:0/4039710263 >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 msgr2=0x7f0a700cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/4039710263 >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 0x7f0a700cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a74bbf640 1 -- 172.21.15.64:0/4039710263 >> v1:172.21.15.100:6789/0 conn(0x7f0a700ccca0 legacy=0x7f0a700d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a74bbf640 1 -- 172.21.15.64:0/4039710263 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a700cd720 con 0x7f0a7015c060 2024-04-01T05:49:04.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.336+0000 7f0a6ffff640 1 --2- 172.21.15.64:0/4039710263 >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 0x7f0a700cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:04.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.338+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/4039710263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 0x7f0a700ce6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009d90 tx=0x7f0a6002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a6dd8807995891 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.339+0000 7f0a6f7fe640 1 -- 172.21.15.64:0/4039710263 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60039070 con 0x7f0a7015c060 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.339+0000 7f0a6f7fe640 1 -- 172.21.15.64:0/4039710263 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6002cce0 con 0x7f0a7015c060 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.339+0000 7f0a6f7fe640 1 -- 172.21.15.64:0/4039710263 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60034660 con 0x7f0a7015c060 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 -- 172.21.15.64:0/4039710263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 msgr2=0x7f0a700ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 --2- 172.21.15.64:0/4039710263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 0x7f0a700ce6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009d90 tx=0x7f0a6002bf30 comp rx=0 tx=0).stop 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 -- 172.21.15.64:0/4039710263 shutdown_connections 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 --2- 172.21.15.64:0/4039710263 >> v2:172.21.15.64:3300/0 conn(0x7f0a700cebf0 0x7f0a700cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 --2- 172.21.15.64:0/4039710263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a7015c060 0x7f0a700ce6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 -- 172.21.15.64:0/4039710263 >> 172.21.15.64:0/4039710263 conn(0x7f0a7009a980 msgr2=0x7f0a7009ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:04.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 -- 172.21.15.64:0/4039710263 shutdown_connections 2024-04-01T05:49:04.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.340+0000 7f0a76e23640 1 -- 172.21.15.64:0/4039710263 wait complete. 2024-04-01T05:49:04.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.343+0000 7f0a76e23640 1 Processor -- start 2024-04-01T05:49:04.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.343+0000 7f0a76e23640 1 -- start start 2024-04-01T05:49:04.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.345+0000 7f0a76e23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 0x7f0a701f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.345+0000 7f0a76e23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a700cebf0 0x7f0a701f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a76e23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 0x7f0a700b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a76e23640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a700cdfc0 con 0x7f0a7015c060 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a76e23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a700cdce0 con 0x7f0a700cebf0 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a76e23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0a700cd120 con 0x7f0a700ccca0 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a753c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 0x7f0a700b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a753c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 0x7f0a700b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52470/0 (socket says 172.21.15.64:52470) 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a753c0640 1 -- 172.21.15.64:0/3921898133 learned_addr learned my addr 172.21.15.64:0/3921898133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 0x7f0a701f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a6ffff640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a700cebf0 0x7f0a701f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.346+0000 7f0a74bbf640 1 -- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a700cebf0 msgr2=0x7f0a701f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a700cebf0 0x7f0a701f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a74bbf640 1 -- 172.21.15.64:0/3921898133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 msgr2=0x7f0a700b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 0x7f0a700b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a74bbf640 1 -- 172.21.15.64:0/3921898133 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7015c5e0 con 0x7f0a700ccca0 2024-04-01T05:49:04.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a74bbf640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 0x7f0a701f20a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a700d0190 tx=0x7f0a60004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:04.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60039050 con 0x7f0a700ccca0 2024-04-01T05:49:04.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a60045070 con 0x7f0a700ccca0 2024-04-01T05:49:04.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.347+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60009450 con 0x7f0a700ccca0 2024-04-01T05:49:04.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.348+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7015c8c0 con 0x7f0a700ccca0 2024-04-01T05:49:04.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.348+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a701f2c60 con 0x7f0a700ccca0 2024-04-01T05:49:04.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.350+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a700688b0 con 0x7f0a700ccca0 2024-04-01T05:49:04.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.351+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f0a6003e870 con 0x7f0a700ccca0 2024-04-01T05:49:04.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.351+0000 7f0a6d7fa640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 0x7f0a5c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:04.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.352+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f0a600ad030 con 0x7f0a700ccca0 2024-04-01T05:49:04.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.352+0000 7f0a6ffff640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 0x7f0a5c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:04.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.353+0000 7f0a6ffff640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 0x7f0a5c06de50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009560 tx=0x7f0a64009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:04.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.356+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f0a6007baa0 con 0x7f0a700ccca0 2024-04-01T05:49:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.885+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f0a700cc890 con 0x7f0a5c06b990 2024-04-01T05:49:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.886+0000 7f0a6d7fa640 1 -- 172.21.15.64:0/3921898133 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0a700cc890 con 0x7f0a5c06b990 2024-04-01T05:49:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2024-04-01T05:49:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 msgr2=0x7f0a5c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 0x7f0a5c06de50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009560 tx=0x7f0a64009190 comp rx=0 tx=0).stop 2024-04-01T05:49:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 msgr2=0x7f0a701f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 0x7f0a701f20a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a700d0190 tx=0x7f0a60004a40 comp rx=0 tx=0).stop 2024-04-01T05:49:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 shutdown_connections 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f0a5c06b990 0x7f0a5c06de50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7015c060 0x7f0a700b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a700cebf0 0x7f0a701f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 --2- 172.21.15.64:0/3921898133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a700ccca0 0x7f0a701f20a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 >> 172.21.15.64:0/3921898133 conn(0x7f0a7009a980 msgr2=0x7f0a7009ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 shutdown_connections 2024-04-01T05:49:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:04.890+0000 7f0a76e23640 1 -- 172.21.15.64:0/3921898133 wait complete. 2024-04-01T05:49:04.915 INFO:tasks.ceph:Scrubbing osd.5 2024-04-01T05:49:04.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:05.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.206+0000 7efd0239a640 1 Processor -- start 2024-04-01T05:49:05.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.206+0000 7efd0239a640 1 -- start start 2024-04-01T05:49:05.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd0239a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcfc15e9b0 0x7efcfc071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd0239a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efcfc15ed90 0x7efcfc160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd0239a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0ae540 con 0x7efcfc15f240 2024-04-01T05:49:05.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd0239a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7efcfc15e540 con 0x7efcfc15e9b0 2024-04-01T05:49:05.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd0239a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efcfc15f960 con 0x7efcfc15ed90 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efcfbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcfc15e9b0 0x7efcfc071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efcfbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcfc15e9b0 0x7efcfc071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52476/0 (socket says 172.21.15.64:52476) 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efcfbfff640 1 -- 172.21.15.64:0/3933130344 learned_addr learned my addr 172.21.15.64:0/3933130344 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efd00937640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efcfc15f240 0x7efcfc160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33848/0 (socket says 172.21.15.64:33848) 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.208+0000 7efcfb7fe640 1 --2- 172.21.15.64:0/3933130344 >> v2:172.21.15.100:3300/0 conn(0x7efcfc15ed90 0x7efcfc160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.209+0000 7efcfbfff640 1 -- 172.21.15.64:0/3933130344 >> v2:172.21.15.100:3300/0 conn(0x7efcfc15ed90 msgr2=0x7efcfc160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:05.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.209+0000 7efcfbfff640 1 --2- 172.21.15.64:0/3933130344 >> v2:172.21.15.100:3300/0 conn(0x7efcfc15ed90 0x7efcfc160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.209+0000 7efcfbfff640 1 -- 172.21.15.64:0/3933130344 >> v1:172.21.15.64:6789/0 conn(0x7efcfc15f240 legacy=0x7efcfc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.209+0000 7efcfbfff640 1 -- 172.21.15.64:0/3933130344 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc15fc40 con 0x7efcfc15e9b0 2024-04-01T05:49:05.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.211+0000 7efcfbfff640 1 --2- 172.21.15.64:0/3933130344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15e9b0 0x7efcfc071ed0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7efcfc06c1f0 tx=0x7efcec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2c6efdeba7e540 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.211+0000 7efcfaffd640 1 -- 172.21.15.64:0/3933130344 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec035020 con 0x7efcfc15e9b0 2024-04-01T05:49:05.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.211+0000 7efcfaffd640 1 -- 172.21.15.64:0/3933130344 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec031870 con 0x7efcfc15e9b0 2024-04-01T05:49:05.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.211+0000 7efcfaffd640 1 -- 172.21.15.64:0/3933130344 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec031c40 con 0x7efcfc15e9b0 2024-04-01T05:49:05.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 -- 172.21.15.64:0/3933130344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15e9b0 msgr2=0x7efcfc071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 --2- 172.21.15.64:0/3933130344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15e9b0 0x7efcfc071ed0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7efcfc06c1f0 tx=0x7efcec02b910 comp rx=0 tx=0).stop 2024-04-01T05:49:05.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 -- 172.21.15.64:0/3933130344 shutdown_connections 2024-04-01T05:49:05.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 --2- 172.21.15.64:0/3933130344 >> v2:172.21.15.100:3300/0 conn(0x7efcfc15ed90 0x7efcfc160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 --2- 172.21.15.64:0/3933130344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15e9b0 0x7efcfc071ed0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 -- 172.21.15.64:0/3933130344 >> 172.21.15.64:0/3933130344 conn(0x7efcfc0d03d0 msgr2=0x7efcfc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:05.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 -- 172.21.15.64:0/3933130344 shutdown_connections 2024-04-01T05:49:05.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.213+0000 7efd0239a640 1 -- 172.21.15.64:0/3933130344 wait complete. 2024-04-01T05:49:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.216+0000 7efd0239a640 1 Processor -- start 2024-04-01T05:49:05.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.216+0000 7efd0239a640 1 -- start start 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc15e9b0 0x7efcfc06b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 0x7efcfc14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcfc15f240 0x7efcfc1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc0ae820 con 0x7efcfc15ed90 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc072410 con 0x7efcfc15e9b0 2024-04-01T05:49:05.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.218+0000 7efd0239a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc0cc980 con 0x7efcfc15f240 2024-04-01T05:49:05.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 0x7efcfc14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 0x7efcfc14fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52482/0 (socket says 172.21.15.64:52482) 2024-04-01T05:49:05.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 -- 172.21.15.64:0/2699663232 learned_addr learned my addr 172.21.15.64:0/2699663232 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:05.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efd00937640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcfc15f240 0x7efcfc1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfbfff640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc15e9b0 0x7efcfc06b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 -- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcfc15f240 msgr2=0x7efcfc1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcfc15f240 0x7efcfc1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 -- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc15e9b0 msgr2=0x7efcfc06b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc15e9b0 0x7efcfc06b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efcfb7fe640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc152d70 con 0x7efcfc15ed90 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.219+0000 7efd00937640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcfc15f240 0x7efcfc1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efcfb7fe640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 0x7efcfc14fe60 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7efcf000e9b0 tx=0x7efcf000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0017070 con 0x7efcfc15ed90 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf000c9a0 con 0x7efcfc15ed90 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf000cd90 con 0x7efcfc15ed90 2024-04-01T05:49:05.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc153050 con 0x7efcfc15ed90 2024-04-01T05:49:05.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.220+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc153610 con 0x7efcfc15ed90 2024-04-01T05:49:05.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.222+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcfc0688b0 con 0x7efcfc15ed90 2024-04-01T05:49:05.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.223+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7efcf001a070 con 0x7efcfc15ed90 2024-04-01T05:49:05.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcf8ff9640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efce806b990 0x7efce806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7efcf008a7b0 con 0x7efcfc15ed90 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcfbfff640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efce806b990 0x7efce806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcf8ff9640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7efce8071850 0x7efce8073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcf0016ec0 con 0x7efce8071850 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.224+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efcf008f2c0 con 0x7efcfc15ed90 2024-04-01T05:49:05.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.225+0000 7efd00937640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7efce8071850 0x7efce8073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.225+0000 7efcfbfff640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efce806b990 0x7efce806de50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efcec036000 tx=0x7efcec03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.225+0000 7efd00937640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7efce8071850 0x7efce8073c70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.226+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== osd.5 v2:172.21.15.100:6800/64687588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcf0016ec0 con 0x7efce8071850 2024-04-01T05:49:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.265+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 --> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efcfc06e9c0 con 0x7efce8071850 2024-04-01T05:49:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.266+0000 7efcf8ff9640 1 -- 172.21.15.64:0/2699663232 <== osd.5 v2:172.21.15.100:6800/64687588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efcfc06e9c0 con 0x7efce8071850 2024-04-01T05:49:05.266 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:05.266 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:05.267 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.266+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7efce8071850 msgr2=0x7efce8073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.266+0000 7efd0239a640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6800/64687588,v1:172.21.15.100:6801/64687588] conn(0x7efce8071850 0x7efce8073c70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efce806b990 msgr2=0x7efce806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7efce806b990 0x7efce806de50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efcec036000 tx=0x7efcec03c040 comp rx=0 tx=0).stop 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 msgr2=0x7efcfc14fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 --2- 172.21.15.64:0/2699663232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc15ed90 0x7efcfc14fe60 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7efcf000e9b0 tx=0x7efcf000ee70 comp rx=0 tx=0).stop 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd00937640 1 -- 172.21.15.64:0/2699663232 reap_dead start 2024-04-01T05:49:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 shutdown_connections 2024-04-01T05:49:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 >> 172.21.15.64:0/2699663232 conn(0x7efcfc0d03d0 msgr2=0x7efcfc0adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 shutdown_connections 2024-04-01T05:49:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.267+0000 7efd0239a640 1 -- 172.21.15.64:0/2699663232 wait complete. 2024-04-01T05:49:05.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-01T05:49:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.570+0000 7fbabf1d9640 1 Processor -- start 2024-04-01T05:49:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.570+0000 7fbabf1d9640 1 -- start start 2024-04-01T05:49:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.571+0000 7fbabf1d9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbab806ff70 0x7fbab8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.571+0000 7fbabf1d9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.571+0000 7fbabf1d9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab80d09b0 con 0x7fbab80cbca0 2024-04-01T05:49:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.571+0000 7fbabf1d9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbab80722a0 con 0x7fbab806ff70 2024-04-01T05:49:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.571+0000 7fbabf1d9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbab8072580 con 0x7fbab8070350 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabd776640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbab80cbca0 0x7fbab806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33856/0 (socket says 172.21.15.64:33856) 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabd776640 1 -- 172.21.15.64:0/32858861 learned_addr learned my addr 172.21.15.64:0/32858861 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabcf75640 1 --2- 172.21.15.64:0/32858861 >> v2:172.21.15.100:3300/0 conn(0x7fbab806ff70 0x7fbab8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbaaffff640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabcf75640 1 -- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 msgr2=0x7fbab80adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabcf75640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabcf75640 1 -- 172.21.15.64:0/32858861 >> v1:172.21.15.64:6789/0 conn(0x7fbab80cbca0 legacy=0x7fbab806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.572+0000 7fbabcf75640 1 -- 172.21.15.64:0/32858861 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab8072860 con 0x7fbab806ff70 2024-04-01T05:49:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.573+0000 7fbabcf75640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8071cc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbab40097a0 tx=0x7fbab402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05a42ea0a58cde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.573+0000 7fbaaf7fe640 1 -- 172.21.15.64:0/32858861 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab40315a0 con 0x7fbab806ff70 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.573+0000 7fbaaf7fe640 1 -- 172.21.15.64:0/32858861 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4037070 con 0x7fbab806ff70 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.573+0000 7fbaaf7fe640 1 -- 172.21.15.64:0/32858861 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4031960 con 0x7fbab806ff70 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 -- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 msgr2=0x7fbab8071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8071cc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbab40097a0 tx=0x7fbab402b840 comp rx=0 tx=0).stop 2024-04-01T05:49:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 -- 172.21.15.64:0/32858861 shutdown_connections 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/32858861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8071cc0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 -- 172.21.15.64:0/32858861 >> 172.21.15.64:0/32858861 conn(0x7fbab80d0c70 msgr2=0x7fbab80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 -- 172.21.15.64:0/32858861 shutdown_connections 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.574+0000 7fbabf1d9640 1 -- 172.21.15.64:0/32858861 wait complete. 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.575+0000 7fbabf1d9640 1 Processor -- start 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.575+0000 7fbabf1d9640 1 -- start start 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 0x7fbab80b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbab806b7d0 con 0x7fbab80cbca0 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbab80ceb70 con 0x7fbab806ff70 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabf1d9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbab806b950 con 0x7fbab8070350 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabd776640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 0x7fbab80b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabd776640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 0x7fbab80b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52496/0 (socket says 172.21.15.64:52496) 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.576+0000 7fbabd776640 1 -- 172.21.15.64:0/3905396624 learned_addr learned my addr 172.21.15.64:0/3905396624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbabcf75640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 -- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 msgr2=0x7fbab8161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8161e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 -- 172.21.15.64:0/3905396624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 msgr2=0x7fbab80b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 0x7fbab80b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 -- 172.21.15.64:0/3905396624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab81fc6f0 con 0x7fbab8070350 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaaffff640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80b58e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e850 tx=0x7fbaa800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8017070 con 0x7fbab8070350 2024-04-01T05:49:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa800c940 con 0x7fbab8070350 2024-04-01T05:49:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.577+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab8162f10 con 0x7fbab8070350 2024-04-01T05:49:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.578+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800ccf0 con 0x7fbab8070350 2024-04-01T05:49:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.578+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81fbc10 con 0x7fbab8070350 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.580+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab80688b0 con 0x7fbab8070350 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.581+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fbaa801a070 con 0x7fbab8070350 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.581+0000 7fbaad7fa640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 0x7fbaa406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.582+0000 7fbabcf75640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 0x7fbaa406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.582+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fbaa808a3d0 con 0x7fbab8070350 2024-04-01T05:49:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.583+0000 7fbabcf75640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 0x7fbaa406de50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbab4036000 tx=0x7fbab403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:05.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:05.586+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fbaa8058c90 con 0x7fbab8070350 2024-04-01T05:49:06.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.114+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fbab8164390 con 0x7fbaa406b990 2024-04-01T05:49:06.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.115+0000 7fbaad7fa640 1 -- 172.21.15.64:0/3905396624 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbab8164390 con 0x7fbaa406b990 2024-04-01T05:49:06.115 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2024-04-01T05:49:06.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 msgr2=0x7fbaa406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 0x7fbaa406de50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbab4036000 tx=0x7fbab403d040 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 msgr2=0x7fbab80b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80b58e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e850 tx=0x7fbaa800ed10 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 shutdown_connections 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fbaa406b990 0x7fbaa406de50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbab80cbca0 0x7fbab80b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbab8070350 0x7fbab80b58e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 --2- 172.21.15.64:0/3905396624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab806ff70 0x7fbab8161e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 >> 172.21.15.64:0/3905396624 conn(0x7fbab80d0c70 msgr2=0x7fbab806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 shutdown_connections 2024-04-01T05:49:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.119+0000 7fbabf1d9640 1 -- 172.21.15.64:0/3905396624 wait complete. 2024-04-01T05:49:06.144 INFO:tasks.ceph:Scrubbing osd.6 2024-04-01T05:49:06.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:06.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.432+0000 7f7d03e8f640 1 Processor -- start 2024-04-01T05:49:06.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.433+0000 7f7d03e8f640 1 -- start start 2024-04-01T05:49:06.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d03e8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7cfc15aca0 0x7f7cfc15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d03e8f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cfc06f5c0 con 0x7f7cfc06ed00 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d03e8f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cfc06f8a0 con 0x7f7cfc072430 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d03e8f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc0701d0 con 0x7f7cfc15aca0 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d0142a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7cfc06ed00 0x7f7cfc071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33862/0 (socket says 172.21.15.64:33862) 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d0142a640 1 -- 172.21.15.64:0/3530839362 learned_addr learned my addr 172.21.15.64:0/3530839362 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.435+0000 7f7d01c2b640 1 --2- 172.21.15.64:0/3530839362 >> v2:172.21.15.100:3300/0 conn(0x7f7cfc15aca0 0x7f7cfc15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450701349 0 0) 0x7f7cfc06f8a0 con 0x7f7cfc072430 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ce8003520 con 0x7f7cfc072430 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763442280 0 0) 0x7f7cfc06f5c0 con 0x7f7cfc06ed00 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d01c2b640 1 -- 172.21.15.64:0/3530839362 >> v1:172.21.15.100:6789/0 conn(0x7f7cfc072430 legacy=0x7f7cfc0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d01c2b640 1 -- 172.21.15.64:0/3530839362 >> v1:172.21.15.64:6789/0 conn(0x7f7cfc06ed00 legacy=0x7f7cfc071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.436+0000 7f7d01c2b640 1 -- 172.21.15.64:0/3530839362 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc06fb80 con 0x7f7cfc15aca0 2024-04-01T05:49:06.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.438+0000 7f7d01c2b640 1 --2- 172.21.15.64:0/3530839362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc15aca0 0x7f7cfc15f2b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80097a0 tx=0x7f7cf802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e255ef3d665b5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.438+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8034070 con 0x7f7cfc15aca0 2024-04-01T05:49:06.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.438+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf8030610 con 0x7f7cfc15aca0 2024-04-01T05:49:06.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.438+0000 7f7d00c29640 1 -- 172.21.15.64:0/3530839362 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80309e0 con 0x7f7cfc15aca0 2024-04-01T05:49:06.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.439+0000 7f7d03e8f640 1 -- 172.21.15.64:0/3530839362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc15aca0 msgr2=0x7f7cfc15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.439+0000 7f7d03e8f640 1 --2- 172.21.15.64:0/3530839362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc15aca0 0x7f7cfc15f2b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80097a0 tx=0x7f7cf802b960 comp rx=0 tx=0).stop 2024-04-01T05:49:06.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.440+0000 7f7d03e8f640 1 -- 172.21.15.64:0/3530839362 shutdown_connections 2024-04-01T05:49:06.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.440+0000 7f7d03e8f640 1 --2- 172.21.15.64:0/3530839362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc15aca0 0x7f7cfc15f2b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.440+0000 7f7d03e8f640 1 -- 172.21.15.64:0/3530839362 >> 172.21.15.64:0/3530839362 conn(0x7f7cfc0d0bd0 msgr2=0x7f7cfc0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:06.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.440+0000 7f7d03e8f640 1 -- 172.21.15.64:0/3530839362 shutdown_connections 2024-04-01T05:49:06.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.440+0000 7f7d03e8f640 1 -- 172.21.15.64:0/3530839362 wait complete. 2024-04-01T05:49:06.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.443+0000 7f7d03e8f640 1 Processor -- start 2024-04-01T05:49:06.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.443+0000 7f7d03e8f640 1 -- start start 2024-04-01T05:49:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cfc06ed00 0x7f7cfc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc072430 0x7f7cfc0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cfc15aca0 0x7f7cfc0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc070660 con 0x7f7cfc06ed00 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc15b080 con 0x7f7cfc072430 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d03e8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc06f190 con 0x7f7cfc15aca0 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d01c2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cfc06ed00 0x7f7cfc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d01c2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cfc06ed00 0x7f7cfc14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52498/0 (socket says 172.21.15.64:52498) 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d01c2b640 1 -- 172.21.15.64:0/2548767465 learned_addr learned my addr 172.21.15.64:0/2548767465 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d0142a640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc072430 0x7f7cfc0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.445+0000 7f7d0242c640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cfc15aca0 0x7f7cfc0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 -- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cfc15aca0 msgr2=0x7f7cfc0c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cfc15aca0 0x7f7cfc0c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 -- 172.21.15.64:0/2548767465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cfc06ed00 msgr2=0x7f7cfc14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cfc06ed00 0x7f7cfc14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc14fd90 con 0x7f7cfc072430 2024-04-01T05:49:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d0142a640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc072430 0x7f7cfc0c6790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c410 tx=0x7f7cec00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0074e0 con 0x7f7cfc072430 2024-04-01T05:49:06.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec0047e0 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc0cc350 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.446+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec004bd0 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.447+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc0cc910 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.448+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cfc0688b0 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.449+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f7cec011070 con 0x7f7cfc072430 2024-04-01T05:49:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.449+0000 7f7cf27fc640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ce006b990 0x7f7ce006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7d01c2b640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ce006b990 0x7f7ce006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f7cec08a050 con 0x7f7cfc072430 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7cf27fc640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f7ce0071850 0x7f7ce0073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cec009e50 con 0x7f7ce0071850 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7cec091050 con 0x7f7cfc072430 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.450+0000 7f7d0242c640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f7ce0071850 0x7f7ce0073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.451+0000 7f7d01c2b640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ce006b990 0x7f7ce006de50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009560 tx=0x7f7cf803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.451+0000 7f7d0242c640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f7ce0071850 0x7f7ce0073c70 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.452+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== osd.6 v2:172.21.15.100:6808/3251071123 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cec009e50 con 0x7f7ce0071850 2024-04-01T05:49:06.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.490+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 --> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7cfc150830 con 0x7f7ce0071850 2024-04-01T05:49:06.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.491+0000 7f7cf27fc640 1 -- 172.21.15.64:0/2548767465 <== osd.6 v2:172.21.15.100:6808/3251071123 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7cfc150830 con 0x7f7ce0071850 2024-04-01T05:49:06.491 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:06.492 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:06.492 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f7ce0071850 msgr2=0x7f7ce0073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6808/3251071123,v1:172.21.15.100:6809/3251071123] conn(0x7f7ce0071850 0x7f7ce0073c70 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ce006b990 msgr2=0x7f7ce006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f7ce006b990 0x7f7ce006de50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009560 tx=0x7f7cf803c040 comp rx=0 tx=0).stop 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc072430 msgr2=0x7f7cfc0c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 --2- 172.21.15.64:0/2548767465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cfc072430 0x7f7cfc0c6790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c410 tx=0x7f7cec00c8d0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d0242c640 1 -- 172.21.15.64:0/2548767465 reap_dead start 2024-04-01T05:49:06.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 shutdown_connections 2024-04-01T05:49:06.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 >> 172.21.15.64:0/2548767465 conn(0x7f7cfc0d0bd0 msgr2=0x7f7cfc0b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:06.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 shutdown_connections 2024-04-01T05:49:06.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.492+0000 7f7d03e8f640 1 -- 172.21.15.64:0/2548767465 wait complete. 2024-04-01T05:49:06.517 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-01T05:49:06.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.796+0000 7fd590127640 1 Processor -- start 2024-04-01T05:49:06.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.797+0000 7fd590127640 1 -- start start 2024-04-01T05:49:06.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd590127640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd58815df10 0x7fd5880d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd590127640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd588162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd590127640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5880afdf0 con 0x7fd58806f810 2024-04-01T05:49:06.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd590127640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd588162b70 con 0x7fd58815df10 2024-04-01T05:49:06.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd590127640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd58806f380 con 0x7fd58806b7d0 2024-04-01T05:49:06.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd58e6c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd588162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd58dec3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd58815df10 0x7fd5880d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd58d6c2640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd58806f810 0x7fd58806b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:55570/0 (socket says 172.21.15.64:55570) 2024-04-01T05:49:06.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.799+0000 7fd58d6c2640 1 -- 172.21.15.64:0/467458255 learned_addr learned my addr 172.21.15.64:0/467458255 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.800+0000 7fd58e6c4640 1 -- 172.21.15.64:0/467458255 >> v2:172.21.15.100:3300/0 conn(0x7fd58815df10 msgr2=0x7fd5880d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.800+0000 7fd58e6c4640 1 --2- 172.21.15.64:0/467458255 >> v2:172.21.15.100:3300/0 conn(0x7fd58815df10 0x7fd5880d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.800+0000 7fd58e6c4640 1 -- 172.21.15.64:0/467458255 >> v1:172.21.15.100:6789/0 conn(0x7fd58806f810 legacy=0x7fd58806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.800+0000 7fd58e6c4640 1 -- 172.21.15.64:0/467458255 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880b00d0 con 0x7fd58806b7d0 2024-04-01T05:49:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.801+0000 7fd58e6c4640 1 --2- 172.21.15.64:0/467458255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd588162630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd58806f500 tx=0x7fd58002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c78e42a77e8529 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.801+0000 7fd58cec1640 1 -- 172.21.15.64:0/467458255 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580039070 con 0x7fd58806b7d0 2024-04-01T05:49:06.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.801+0000 7fd58cec1640 1 -- 172.21.15.64:0/467458255 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58002cc40 con 0x7fd58806b7d0 2024-04-01T05:49:06.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.801+0000 7fd58cec1640 1 -- 172.21.15.64:0/467458255 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580034440 con 0x7fd58806b7d0 2024-04-01T05:49:06.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.802+0000 7fd590127640 1 -- 172.21.15.64:0/467458255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 msgr2=0x7fd588162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.802+0000 7fd590127640 1 --2- 172.21.15.64:0/467458255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd588162630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd58806f500 tx=0x7fd58002be90 comp rx=0 tx=0).stop 2024-04-01T05:49:06.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.802+0000 7fd590127640 1 -- 172.21.15.64:0/467458255 shutdown_connections 2024-04-01T05:49:06.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.802+0000 7fd590127640 1 --2- 172.21.15.64:0/467458255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd588162630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.803+0000 7fd590127640 1 --2- 172.21.15.64:0/467458255 >> v2:172.21.15.100:3300/0 conn(0x7fd58815df10 0x7fd5880d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.803+0000 7fd590127640 1 -- 172.21.15.64:0/467458255 >> 172.21.15.64:0/467458255 conn(0x7fd5880d0c40 msgr2=0x7fd5880d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:06.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.803+0000 7fd590127640 1 -- 172.21.15.64:0/467458255 shutdown_connections 2024-04-01T05:49:06.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.803+0000 7fd590127640 1 -- 172.21.15.64:0/467458255 wait complete. 2024-04-01T05:49:06.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.806+0000 7fd590127640 1 Processor -- start 2024-04-01T05:49:06.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.806+0000 7fd590127640 1 -- start start 2024-04-01T05:49:06.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd58815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 0x7fd58815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd58815e2f0 con 0x7fd58815df10 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880d0930 con 0x7fd58806f810 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd590127640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd58815e470 con 0x7fd58806b7d0 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd58e6c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd58e6c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52500/0 (socket says 172.21.15.64:52500) 2024-04-01T05:49:06.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd58e6c4640 1 -- 172.21.15.64:0/1821863007 learned_addr learned my addr 172.21.15.64:0/1821863007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:06.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd58d6c2640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 0x7fd58815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.808+0000 7fd58dec3640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd58815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 -- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 msgr2=0x7fd58815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd58815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 -- 172.21.15.64:0/1821863007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 msgr2=0x7fd5880c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:06.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 -- 172.21.15.64:0/1821863007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880c6ee0 con 0x7fd58806f810 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58e6c4640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58dec3640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd58815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.809+0000 7fd58d6c2640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 0x7fd58815c950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c370 tx=0x7fd58400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.810+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584017070 con 0x7fd58806f810 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.810+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd584005ce0 con 0x7fd58806f810 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.810+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58815e8a0 con 0x7fd58806f810 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.810+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584007310 con 0x7fd58806f810 2024-04-01T05:49:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.810+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58815ce90 con 0x7fd58806f810 2024-04-01T05:49:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.813+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5880688b0 con 0x7fd58806f810 2024-04-01T05:49:06.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.814+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd58401a070 con 0x7fd58806f810 2024-04-01T05:49:06.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.815+0000 7fd57a7fc640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 0x7fd56c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.815+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd5840897c0 con 0x7fd58806f810 2024-04-01T05:49:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.815+0000 7fd58dec3640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 0x7fd56c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.816+0000 7fd58dec3640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 0x7fd56c06de50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009e90 tx=0x7fd57c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:06.820+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fd584058080 con 0x7fd58806f810 2024-04-01T05:49:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.351+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd58806bec0 con 0x7fd56c06b990 2024-04-01T05:49:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.351+0000 7fd57a7fc640 1 -- 172.21.15.64:0/1821863007 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd58806bec0 con 0x7fd56c06b990 2024-04-01T05:49:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2024-04-01T05:49:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 msgr2=0x7fd56c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 0x7fd56c06de50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009e90 tx=0x7fd57c009190 comp rx=0 tx=0).stop 2024-04-01T05:49:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 msgr2=0x7fd58815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 0x7fd58815c950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c370 tx=0x7fd58400c830 comp rx=0 tx=0).stop 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 shutdown_connections 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd56c06b990 0x7fd56c06de50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd58815df10 0x7fd5880c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58806f810 0x7fd58815c950 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.355+0000 7fd590127640 1 --2- 172.21.15.64:0/1821863007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58806b7d0 0x7fd58815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.356+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 >> 172.21.15.64:0/1821863007 conn(0x7fd5880d0c40 msgr2=0x7fd5880b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.356+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 shutdown_connections 2024-04-01T05:49:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.356+0000 7fd590127640 1 -- 172.21.15.64:0/1821863007 wait complete. 2024-04-01T05:49:07.380 INFO:tasks.ceph:Scrubbing osd.7 2024-04-01T05:49:07.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-01T05:49:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.669+0000 7fd419efb640 1 Processor -- start 2024-04-01T05:49:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.669+0000 7fd419efb640 1 -- start start 2024-04-01T05:49:07.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd419efb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd41415e3c0 0x7fd4140cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd419efb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4140b3f10 con 0x7fd4140b3ac0 2024-04-01T05:49:07.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd419efb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4140cfe30 con 0x7fd4140af970 2024-04-01T05:49:07.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd419efb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7fd41406f860 con 0x7fd41415e3c0 2024-04-01T05:49:07.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd4137fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd41415e3c0 0x7fd4140cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd413fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd4140b3ac0 0x7fd414164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33876/0 (socket says 172.21.15.64:33876) 2024-04-01T05:49:07.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.672+0000 7fd413fff640 1 -- 172.21.15.64:0/1302013386 learned_addr learned my addr 172.21.15.64:0/1302013386 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:07.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.673+0000 7fd4127fc640 1 -- 172.21.15.64:0/1302013386 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839409598 0 0) 0x7fd4140cfe30 con 0x7fd4140af970 2024-04-01T05:49:07.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.673+0000 7fd4127fc640 1 -- 172.21.15.64:0/1302013386 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3fc003520 con 0x7fd4140af970 2024-04-01T05:49:07.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.673+0000 7fd4137fe640 1 -- 172.21.15.64:0/1302013386 >> v1:172.21.15.100:6789/0 conn(0x7fd4140af970 legacy=0x7fd4140b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.673+0000 7fd4137fe640 1 -- 172.21.15.64:0/1302013386 >> v1:172.21.15.64:6789/0 conn(0x7fd4140b3ac0 legacy=0x7fd414164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.673+0000 7fd4137fe640 1 -- 172.21.15.64:0/1302013386 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4140cddc0 con 0x7fd41415e3c0 2024-04-01T05:49:07.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.675+0000 7fd4137fe640 1 --2- 172.21.15.64:0/1302013386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd4140cf8f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd4040097a0 tx=0x7fd40402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3a413a2c3b4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:07.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.675+0000 7fd4127fc640 1 -- 172.21.15.64:0/1302013386 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404034070 con 0x7fd41415e3c0 2024-04-01T05:49:07.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.675+0000 7fd4127fc640 1 -- 172.21.15.64:0/1302013386 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd404030610 con 0x7fd41415e3c0 2024-04-01T05:49:07.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.675+0000 7fd4127fc640 1 -- 172.21.15.64:0/1302013386 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040309e0 con 0x7fd41415e3c0 2024-04-01T05:49:07.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 -- 172.21.15.64:0/1302013386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 msgr2=0x7fd4140cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 --2- 172.21.15.64:0/1302013386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd4140cf8f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd4040097a0 tx=0x7fd40402b960 comp rx=0 tx=0).stop 2024-04-01T05:49:07.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 -- 172.21.15.64:0/1302013386 shutdown_connections 2024-04-01T05:49:07.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 --2- 172.21.15.64:0/1302013386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd4140cf8f0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 -- 172.21.15.64:0/1302013386 >> 172.21.15.64:0/1302013386 conn(0x7fd4140d03c0 msgr2=0x7fd4140d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:07.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 -- 172.21.15.64:0/1302013386 shutdown_connections 2024-04-01T05:49:07.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.677+0000 7fd419efb640 1 -- 172.21.15.64:0/1302013386 wait complete. 2024-04-01T05:49:07.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.680+0000 7fd419efb640 1 Processor -- start 2024-04-01T05:49:07.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.680+0000 7fd419efb640 1 -- start start 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 0x7fd414144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4140b3ac0 0x7fd4141567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd414071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4140cf0f0 con 0x7fd41415e3c0 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4140cee10 con 0x7fd4140b3ac0 2024-04-01T05:49:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.682+0000 7fd419efb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd4140cf3d0 con 0x7fd4140af970 2024-04-01T05:49:07.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 0x7fd414144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd413fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd414071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd412ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4140b3ac0 0x7fd4141567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 0x7fd414144d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.64:42100/0 (socket says 172.21.15.64:42100) 2024-04-01T05:49:07.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 -- 172.21.15.64:0/1433122891 learned_addr learned my addr 172.21.15.64:0/1433122891 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 -- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4140b3ac0 msgr2=0x7fd4141567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4140b3ac0 0x7fd4141567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 -- 172.21.15.64:0/1433122891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 msgr2=0x7fd414071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd41415e3c0 0x7fd414071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd4137fe640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4141571b0 con 0x7fd4140af970 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.683+0000 7fd412ffd640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4140b3ac0 0x7fd4141567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.684+0000 7fd4137fe640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 0x7fd414144d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4040098c0 tx=0x7fd4040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.684+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404034040 con 0x7fd4140af970 2024-04-01T05:49:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.684+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40403aaa0 con 0x7fd4140af970 2024-04-01T05:49:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.684+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040443f0 con 0x7fd4140af970 2024-04-01T05:49:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.684+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41414ef10 con 0x7fd4140af970 2024-04-01T05:49:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.685+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41414f4d0 con 0x7fd4140af970 2024-04-01T05:49:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.686+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd41414f8b0 con 0x7fd4140af970 2024-04-01T05:49:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.687+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fd40404b060 con 0x7fd4140af970 2024-04-01T05:49:07.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.688+0000 7fd3fbfff640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd40006b990 0x7fd40006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd4040ac150 con 0x7fd4140af970 2024-04-01T05:49:07.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd412ffd640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd40006b990 0x7fd40006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd3fbfff640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fd400071850 0x7fd400073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd404001d30 con 0x7fd400071850 2024-04-01T05:49:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd4040b3050 con 0x7fd4140af970 2024-04-01T05:49:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.689+0000 7fd413fff640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fd400071850 0x7fd400073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.690+0000 7fd412ffd640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd40006b990 0x7fd40006de50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd408007fd0 tx=0x7fd408009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.690+0000 7fd413fff640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fd400071850 0x7fd400073c70 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.691+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== osd.7 v2:172.21.15.100:6824/2776403688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd404001d30 con 0x7fd400071850 2024-04-01T05:49:07.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.730+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 --> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd4140688b0 con 0x7fd400071850 2024-04-01T05:49:07.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.730+0000 7fd3fbfff640 1 -- 172.21.15.64:0/1433122891 <== osd.7 v2:172.21.15.100:6824/2776403688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd4140688b0 con 0x7fd400071850 2024-04-01T05:49:07.731 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-01T05:49:07.731 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-01T05:49:07.731 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-01T05:49:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fd400071850 msgr2=0x7fd400073c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6824/2776403688,v1:172.21.15.100:6825/2776403688] conn(0x7fd400071850 0x7fd400073c70 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd40006b990 msgr2=0x7fd40006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fd40006b990 0x7fd40006de50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd408007fd0 tx=0x7fd408009040 comp rx=0 tx=0).stop 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 msgr2=0x7fd414144d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 --2- 172.21.15.64:0/1433122891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4140af970 0x7fd414144d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4040098c0 tx=0x7fd4040046c0 comp rx=0 tx=0).stop 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd413fff640 1 -- 172.21.15.64:0/1433122891 reap_dead start 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 shutdown_connections 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.731+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 >> 172.21.15.64:0/1433122891 conn(0x7fd4140d03c0 msgr2=0x7fd4140b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.732+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 shutdown_connections 2024-04-01T05:49:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:07.732+0000 7fd419efb640 1 -- 172.21.15.64:0/1433122891 wait complete. 2024-04-01T05:49:07.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-01T05:49:08.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.036+0000 7f5ac609d640 1 Processor -- start 2024-04-01T05:49:08.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.036+0000 7f5ac609d640 1 -- start start 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5ac609d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ac015c070 0x7f5ac00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5ac609d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ac00cedb0 0x7f5ac00c6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5ac609d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac00d0770 con 0x7f5ac00d02b0 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5ac609d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac00d0a50 con 0x7f5ac015c070 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5ac609d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac00c5a30 con 0x7f5ac00cedb0 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5abf7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ac015c070 0x7f5ac00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5abeffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5ac00d02b0 0x7f5ac00ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33890/0 (socket says 172.21.15.64:33890) 2024-04-01T05:49:08.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.038+0000 7f5abeffd640 1 -- 172.21.15.64:0/2960530557 learned_addr learned my addr 172.21.15.64:0/2960530557 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:08.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abffff640 1 --2- 172.21.15.64:0/2960530557 >> v2:172.21.15.100:3300/0 conn(0x7f5ac00cedb0 0x7f5ac00c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abf7fe640 1 -- 172.21.15.64:0/2960530557 >> v2:172.21.15.100:3300/0 conn(0x7f5ac00cedb0 msgr2=0x7f5ac00c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:08.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abf7fe640 1 --2- 172.21.15.64:0/2960530557 >> v2:172.21.15.100:3300/0 conn(0x7f5ac00cedb0 0x7f5ac00c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abf7fe640 1 -- 172.21.15.64:0/2960530557 >> v1:172.21.15.64:6789/0 conn(0x7f5ac00d02b0 legacy=0x7f5ac00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abf7fe640 1 -- 172.21.15.64:0/2960530557 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac00c5d10 con 0x7f5ac015c070 2024-04-01T05:49:08.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.039+0000 7f5abe7fc640 1 -- 172.21.15.64:0/2960530557 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806697410 0 0) 0x7f5ac00d0770 con 0x7f5ac00d02b0 2024-04-01T05:49:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.041+0000 7f5abf7fe640 1 --2- 172.21.15.64:0/2960530557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac00d27f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0072c10 tx=0x7f5ab002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c8bcdabadf914 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.042+0000 7f5abe7fc640 1 -- 172.21.15.64:0/2960530557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0035020 con 0x7f5ac015c070 2024-04-01T05:49:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.042+0000 7f5abe7fc640 1 -- 172.21.15.64:0/2960530557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0031830 con 0x7f5ac015c070 2024-04-01T05:49:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.042+0000 7f5abe7fc640 1 -- 172.21.15.64:0/2960530557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0031c00 con 0x7f5ac015c070 2024-04-01T05:49:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 -- 172.21.15.64:0/2960530557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 msgr2=0x7f5ac00d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 --2- 172.21.15.64:0/2960530557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac00d27f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0072c10 tx=0x7f5ab002b8d0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 -- 172.21.15.64:0/2960530557 shutdown_connections 2024-04-01T05:49:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 --2- 172.21.15.64:0/2960530557 >> v2:172.21.15.100:3300/0 conn(0x7f5ac00cedb0 0x7f5ac00c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 --2- 172.21.15.64:0/2960530557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac00d27f0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 -- 172.21.15.64:0/2960530557 >> 172.21.15.64:0/2960530557 conn(0x7f5ac00d1cd0 msgr2=0x7f5ac00d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 -- 172.21.15.64:0/2960530557 shutdown_connections 2024-04-01T05:49:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.043+0000 7f5ac609d640 1 -- 172.21.15.64:0/2960530557 wait complete. 2024-04-01T05:49:08.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.046+0000 7f5ac609d640 1 Processor -- start 2024-04-01T05:49:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.047+0000 7f5ac609d640 1 -- start start 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 0x7f5ac015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac00d02b0 0x7f5ac009c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00c6770 con 0x7f5ac015c070 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac015c450 con 0x7f5ac00d02b0 2024-04-01T05:49:08.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5ac609d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac00c68f0 con 0x7f5ac00cedb0 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5abffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5abf7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 0x7f5ac015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5abffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52546/0 (socket says 172.21.15.64:52546) 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5abeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac00d02b0 0x7f5ac009c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.049+0000 7f5abffff640 1 -- 172.21.15.64:0/3864457405 learned_addr learned my addr 172.21.15.64:0/3864457405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 -- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac00d02b0 msgr2=0x7f5ac009c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac00d02b0 0x7f5ac009c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 -- 172.21.15.64:0/3864457405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 msgr2=0x7f5ac015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac015e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 -- 172.21.15.64:0/3864457405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac00b3170 con 0x7f5ac00cedb0 2024-04-01T05:49:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5abf7fe640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 0x7f5ac015a740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0033000 tx=0x7f5ab00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.050+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0035040 con 0x7f5ac00cedb0 2024-04-01T05:49:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.051+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0004230 con 0x7f5ac00cedb0 2024-04-01T05:49:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.051+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac009fac0 con 0x7f5ac00cedb0 2024-04-01T05:49:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.051+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab003a700 con 0x7f5ac00cedb0 2024-04-01T05:49:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.051+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac009ec60 con 0x7f5ac00cedb0 2024-04-01T05:49:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.054+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac009efa0 con 0x7f5ac00cedb0 2024-04-01T05:49:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.054+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f5ab0038020 con 0x7f5ac00cedb0 2024-04-01T05:49:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.055+0000 7f5aa7fff640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 0x7f5aac06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.055+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5ab00ac0b0 con 0x7f5ac00cedb0 2024-04-01T05:49:08.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.055+0000 7f5abeffd640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 0x7f5aac06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.056+0000 7f5abeffd640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 0x7f5aac06de50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0070420 tx=0x7f5ab400e540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.060+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f5ac009efa0 con 0x7f5ac00cedb0 2024-04-01T05:49:08.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.591+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f5ac015bbb0 con 0x7f5aac06b990 2024-04-01T05:49:08.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.591+0000 7f5aa7fff640 1 -- 172.21.15.64:0/3864457405 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5ac015bbb0 con 0x7f5aac06b990 2024-04-01T05:49:08.594 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2024-04-01T05:49:08.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 msgr2=0x7f5aac06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 0x7f5aac06de50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0070420 tx=0x7f5ab400e540 comp rx=0 tx=0).stop 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 msgr2=0x7f5ac015a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 0x7f5ac015a740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0033000 tx=0x7f5ab00046c0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 shutdown_connections 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f5aac06b990 0x7f5aac06de50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac015c070 0x7f5ac015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac00d02b0 0x7f5ac009c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 --2- 172.21.15.64:0/3864457405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ac00cedb0 0x7f5ac015a740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 >> 172.21.15.64:0/3864457405 conn(0x7f5ac00d1cd0 msgr2=0x7f5ac006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:08.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 shutdown_connections 2024-04-01T05:49:08.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.597+0000 7f5ac609d640 1 -- 172.21.15.64:0/3864457405 wait complete. 2024-04-01T05:49:08.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T05:49:08.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.897+0000 7ffa8fc72640 1 Processor -- start 2024-04-01T05:49:08.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.897+0000 7ffa8fc72640 1 -- start start 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.899+0000 7ffa8fc72640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffa8806b0c0 0x7ffa8815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.899+0000 7ffa8fc72640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8806b570 con 0x7ffa880cc770 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.899+0000 7ffa8fc72640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8806b7f0 con 0x7ffa8815eb90 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.899+0000 7ffa8fc72640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ffa8806bad0 con 0x7ffa8806b0c0 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8da0e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffa8815eb90 0x7ffa88162510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:55590/0 (socket says 172.21.15.64:55590) 2024-04-01T05:49:08.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8da0e640 1 -- 172.21.15.64:0/3418048691 learned_addr learned my addr 172.21.15.64:0/3418048691 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8d20d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffa8806b0c0 0x7ffa8815ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8e20f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa880cc770 0x7ffa880ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33900/0 (socket says 172.21.15.64:33900) 2024-04-01T05:49:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8d20d640 1 -- 172.21.15.64:0/3418048691 >> v1:172.21.15.100:6789/0 conn(0x7ffa8815eb90 legacy=0x7ffa88162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8ca0c640 1 -- 172.21.15.64:0/3418048691 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169505622 0 0) 0x7ffa8806b7f0 con 0x7ffa8815eb90 2024-04-01T05:49:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8d20d640 1 -- 172.21.15.64:0/3418048691 >> v1:172.21.15.64:6789/0 conn(0x7ffa880cc770 legacy=0x7ffa880ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.900+0000 7ffa8d20d640 1 -- 172.21.15.64:0/3418048691 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa88161d30 con 0x7ffa8806b0c0 2024-04-01T05:49:08.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.902+0000 7ffa8d20d640 1 --2- 172.21.15.64:0/3418048691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8806b0c0 0x7ffa8815ef70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097a0 tx=0x7ffa7802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347219fd0c48b33a server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.902+0000 7ffa8ca0c640 1 -- 172.21.15.64:0/3418048691 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78037070 con 0x7ffa8806b0c0 2024-04-01T05:49:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.903+0000 7ffa8ca0c640 1 -- 172.21.15.64:0/3418048691 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa78030710 con 0x7ffa8806b0c0 2024-04-01T05:49:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.903+0000 7ffa8ca0c640 1 -- 172.21.15.64:0/3418048691 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78030ae0 con 0x7ffa8806b0c0 2024-04-01T05:49:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 -- 172.21.15.64:0/3418048691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8806b0c0 msgr2=0x7ffa8815ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/3418048691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8806b0c0 0x7ffa8815ef70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097a0 tx=0x7ffa7802b790 comp rx=0 tx=0).stop 2024-04-01T05:49:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 -- 172.21.15.64:0/3418048691 shutdown_connections 2024-04-01T05:49:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/3418048691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8806b0c0 0x7ffa8815ef70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 -- 172.21.15.64:0/3418048691 >> 172.21.15.64:0/3418048691 conn(0x7ffa880d0be0 msgr2=0x7ffa880d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 -- 172.21.15.64:0/3418048691 shutdown_connections 2024-04-01T05:49:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.904+0000 7ffa8fc72640 1 -- 172.21.15.64:0/3418048691 wait complete. 2024-04-01T05:49:08.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.907+0000 7ffa8fc72640 1 Processor -- start 2024-04-01T05:49:08.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.907+0000 7ffa8fc72640 1 -- start start 2024-04-01T05:49:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.909+0000 7ffa8fc72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 0x7ffa880b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.909+0000 7ffa8fc72640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa880cc770 0x7ffa880c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.909+0000 7ffa8fc72640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 0x7ffa880c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8fc72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8815f990 con 0x7ffa8806b0c0 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8fc72640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8815f4b0 con 0x7ffa8815eb90 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8fc72640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8815fb10 con 0x7ffa880cc770 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8da0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 0x7ffa880b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8da0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 0x7ffa880b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52562/0 (socket says 172.21.15.64:52562) 2024-04-01T05:49:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8da0e640 1 -- 172.21.15.64:0/2377880172 learned_addr learned my addr 172.21.15.64:0/2377880172 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 0x7ffa880c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8d20d640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa880cc770 0x7ffa880c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 -- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa880cc770 msgr2=0x7ffa880c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa880cc770 0x7ffa880c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 -- 172.21.15.64:0/2377880172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 msgr2=0x7ffa880b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 0x7ffa880b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.910+0000 7ffa8e20f640 1 -- 172.21.15.64:0/2377880172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa880c62e0 con 0x7ffa8815eb90 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa8e20f640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 0x7ffa880c67a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ffa80002230 tx=0x7ffa8000cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80018070 con 0x7ffa8815eb90 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8000a820 con 0x7ffa8815eb90 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000abf0 con 0x7ffa8815eb90 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa880b4b00 con 0x7ffa8815eb90 2024-04-01T05:49:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.911+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa880b50c0 con 0x7ffa8815eb90 2024-04-01T05:49:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.913+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ffa8001b070 con 0x7ffa8815eb90 2024-04-01T05:49:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.913+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa880688b0 con 0x7ffa8815eb90 2024-04-01T05:49:08.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.914+0000 7ffa7e7fc640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 0x7ffa6c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:08.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.914+0000 7ffa8da0e640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 0x7ffa6c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:08.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.915+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ffa80089df0 con 0x7ffa8815eb90 2024-04-01T05:49:08.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.915+0000 7ffa8da0e640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 0x7ffa6c06de80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffa84000f60 tx=0x7ffa84007240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:08.919+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ffa8806b570 con 0x7ffa8815eb90 2024-04-01T05:49:09.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.438+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa880b4820 con 0x7ffa6c06b9c0 2024-04-01T05:49:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.439+0000 7ffa7e7fc640 1 -- 172.21.15.64:0/2377880172 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13447 (secure 0 0 0) 0x7ffa880b4820 con 0x7ffa6c06b9c0 2024-04-01T05:49:09.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:49:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 msgr2=0x7ffa6c06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 0x7ffa6c06de80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffa84000f60 tx=0x7ffa84007240 comp rx=0 tx=0).stop 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 msgr2=0x7ffa880c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 0x7ffa880c67a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ffa80002230 tx=0x7ffa8000cf50 comp rx=0 tx=0).stop 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 shutdown_connections 2024-04-01T05:49:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ffa6c06b9c0 0x7ffa6c06de80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8815eb90 0x7ffa880c67a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa880cc770 0x7ffa880c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 --2- 172.21.15.64:0/2377880172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa8806b0c0 0x7ffa880b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 >> 172.21.15.64:0/2377880172 conn(0x7ffa880d0be0 msgr2=0x7ffa880d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.444+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 shutdown_connections 2024-04-01T05:49:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:09.445+0000 7ffa8fc72640 1 -- 172.21.15.64:0/2377880172 wait complete. 2024-04-01T05:49:09.470 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10094,"stamp":"2024-04-01T05:49:08.875863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":12552332,"kb_used_data":4424,"kb_used_omap":5092,"kb_used_meta":12542747,"kb_avail":742422388,"statfs":{"total":773094113280,"available":760240525312,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5215210,"internal_metadata":12843772950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002655"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":367,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-01T05:48:49.085444+0000","last_change":"2024-04-01T00:13:36.815724+0000","last_active":"2024-04-01T05:48:49.085444+0000","last_peered":"2024-04-01T05:48:49.085444+0000","last_clean":"2024-04-01T05:48:49.085444+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T05:48:49.085444+0000","last_undegraded":"2024-04-01T05:48:49.085444+0000","last_fullsized":"2024-04-01T05:48:49.085444+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-01T00:13:34.825848+0000","last_clean_scrub_stamp":"2024-04-01T00:13:34.825848+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T07:32:16.411838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834578876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1635812,"kb_used_data":988,"kb_used_omap":388,"kb_used_meta":1634427,"kb_avail":92736028,"statfs":{"total":96636764160,"available":94961692672,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":398295,"internal_metadata":1673653289},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834578877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1626080,"kb_used_data":408,"kb_used_omap":406,"kb_used_meta":1625257,"kb_avail":92745760,"statfs":{"total":96636764160,"available":94971658240,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":416539,"internal_metadata":1664263397},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834578878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1789732,"kb_used_data":408,"kb_used_omap":144,"kb_used_meta":1789167,"kb_avail":92582108,"statfs":{"total":96636764160,"available":94804078592,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":147804,"internal_metadata":1832107684},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834578875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1805024,"kb_used_data":408,"kb_used_omap":839,"kb_used_meta":1803768,"kb_avail":92566816,"statfs":{"total":96636764160,"available":94788419584,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":859451,"internal_metadata":1847059141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834578837,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1174048,"kb_used_data":408,"kb_used_omap":1243,"kb_used_meta":1172388,"kb_avail":93197792,"statfs":{"total":96636764160,"available":95434539008,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1273266,"internal_metadata":1200525902},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834578856,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1320224,"kb_used_data":408,"kb_used_omap":315,"kb_used_meta":1319492,"kb_avail":93051616,"statfs":{"total":96636764160,"available":95284854784,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":323406,"internal_metadata":1351159986},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834578850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1694944,"kb_used_data":408,"kb_used_omap":1472,"kb_used_meta":1693055,"kb_avail":92676896,"statfs":{"total":96636764160,"available":94901141504,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507447,"internal_metadata":1733689225},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834578870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1506468,"kb_used_data":988,"kb_used_omap":282,"kb_used_meta":1505189,"kb_avail":92865372,"statfs":{"total":96636764160,"available":95094140928,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":289002,"internal_metadata":1541314326},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T05:49:09.473 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-01T00:13:34.825848+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=1, tm_hour=0, tm_min=13, tm_sec=34, tm_wday=0, tm_yday=92, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=1, tm_hour=5, tm_min=48, tm_sec=57, tm_wday=0, tm_yday=92, tm_isdst=0) 2024-04-01T05:49:09.473 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-01T05:49:29.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-01T05:49:29.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.751+0000 7fb9b430d640 1 Processor -- start 2024-04-01T05:49:29.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.752+0000 7fb9b430d640 1 -- start start 2024-04-01T05:49:29.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.754+0000 7fb9b430d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 0x7fb9ac15e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:29.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.754+0000 7fb9b430d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9ac0af3d0 con 0x7fb9ac0ce5a0 2024-04-01T05:49:29.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.754+0000 7fb9b430d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9ac0af6b0 con 0x7fb9ac0ce1c0 2024-04-01T05:49:29.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.754+0000 7fb9b430d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac0af990 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b20a9640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb9ac0ce1c0 0x7fb9ac0aeb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:40720/0 (socket says 172.21.15.64:40720) 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b20a9640 1 -- 172.21.15.64:0/821468824 learned_addr learned my addr 172.21.15.64:0/821468824 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/821468824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 0x7fb9ac15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1179995294 0 0) 0x7fb9ac0af3d0 con 0x7fb9ac0ce5a0 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb998003520 con 0x7fb9ac0ce5a0 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453301479 0 0) 0x7fb9ac0af6b0 con 0x7fb9ac0ce1c0 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9ac0af3d0 con 0x7fb9ac0ce1c0 2024-04-01T05:49:29.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b18a8640 1 -- 172.21.15.64:0/821468824 >> v1:172.21.15.100:6789/0 conn(0x7fb9ac0ce1c0 legacy=0x7fb9ac0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:29.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b18a8640 1 -- 172.21.15.64:0/821468824 >> v1:172.21.15.64:6789/0 conn(0x7fb9ac0ce5a0 legacy=0x7fb9ac15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:29.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.755+0000 7fb9b18a8640 1 -- 172.21.15.64:0/821468824 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac06d6e0 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.757+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/821468824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 0x7fb9ac15e620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097d0 tx=0x7fb99c02b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b7b38a0ffd45c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:29.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.758+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c034070 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.758+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c030890 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.758+0000 7fb9b10a7640 1 -- 172.21.15.64:0/821468824 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c030c40 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 -- 172.21.15.64:0/821468824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 msgr2=0x7fb9ac15e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:29.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 --2- 172.21.15.64:0/821468824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 0x7fb9ac15e620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097d0 tx=0x7fb99c02b9d0 comp rx=0 tx=0).stop 2024-04-01T05:49:29.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 -- 172.21.15.64:0/821468824 shutdown_connections 2024-04-01T05:49:29.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 --2- 172.21.15.64:0/821468824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0c3a10 0x7fb9ac15e620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:29.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 -- 172.21.15.64:0/821468824 >> 172.21.15.64:0/821468824 conn(0x7fb9ac0d0380 msgr2=0x7fb9ac0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:29.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.759+0000 7fb9b430d640 1 -- 172.21.15.64:0/821468824 shutdown_connections 2024-04-01T05:49:29.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.760+0000 7fb9b430d640 1 -- 172.21.15.64:0/821468824 wait complete. 2024-04-01T05:49:29.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.763+0000 7fb9b430d640 1 Processor -- start 2024-04-01T05:49:29.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.763+0000 7fb9b430d640 1 -- start start 2024-04-01T05:49:29.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:29.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 0x7fb9ac0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac0ce5a0 0x7fb9ac159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac06ed90 con 0x7fb9ac0ce5a0 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac06ec10 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b430d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac06ef10 con 0x7fb9ac0ce1c0 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b20a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b20a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.64:56500/0 (socket says 172.21.15.64:56500) 2024-04-01T05:49:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b20a9640 1 -- 172.21.15.64:0/4027910144 learned_addr learned my addr 172.21.15.64:0/4027910144 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b28aa640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac0ce5a0 0x7fb9ac159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.765+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 0x7fb9ac0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 -- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 msgr2=0x7fb9ac0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 0x7fb9ac0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 -- 172.21.15.64:0/4027910144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac0ce5a0 msgr2=0x7fb9ac159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac0ce5a0 0x7fb9ac159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 -- 172.21.15.64:0/4027910144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac0cc560 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 0x7fb9ac0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b20a9640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c500 tx=0x7fb9a800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8017070 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a8005ce0 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8007330 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac15a0c0 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.766+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac0cade0 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.768+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7fb9a801a070 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.768+0000 7fb9a2ffd640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 0x7fb99006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:29.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.769+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb9a8089950 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.769+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 0x7fb99006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:29.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.769+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ac0cec50 con 0x7fb9ac0c3a10 2024-04-01T05:49:29.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.770+0000 7fb9b18a8640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 0x7fb99006de50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb99c036000 tx=0x7fb99c03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:29.774+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7fb9ac0cec50 con 0x7fb9ac0c3a10 2024-04-01T05:49:30.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.300+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 --> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9ac0ca8b0 con 0x7fb99006b990 2024-04-01T05:49:30.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.301+0000 7fb9a2ffd640 1 -- 172.21.15.64:0/4027910144 <== mgr.4108 v2:172.21.15.100:6832/93206356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13455 (secure 0 0 0) 0x7fb9ac0ca8b0 con 0x7fb99006b990 2024-04-01T05:49:30.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:49:30.303 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-04-01T05:49:30.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 msgr2=0x7fb99006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:30.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 0x7fb99006de50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb99c036000 tx=0x7fb99c03b040 comp rx=0 tx=0).stop 2024-04-01T05:49:30.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 msgr2=0x7fb9ac0cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:30.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c500 tx=0x7fb9a800c9c0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 shutdown_connections 2024-04-01T05:49:30.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7fb99006b990 0x7fb99006de50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac0ce5a0 0x7fb9ac159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9ac0ce1c0 0x7fb9ac0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 --2- 172.21.15.64:0/4027910144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9ac0c3a10 0x7fb9ac0cd400 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 >> 172.21.15.64:0/4027910144 conn(0x7fb9ac0d0380 msgr2=0x7fb9ac0b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 shutdown_connections 2024-04-01T05:49:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.305+0000 7fb9b430d640 1 -- 172.21.15.64:0/4027910144 wait complete. 2024-04-01T05:49:30.331 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10104,"stamp":"2024-04-01T05:49:28.880074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":12552332,"kb_used_data":4424,"kb_used_omap":5092,"kb_used_meta":12542747,"kb_avail":742422388,"statfs":{"total":773094113280,"available":760240525312,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5215210,"internal_metadata":12843772950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002437"},"pg_stats":[{"pgid":"1.0","version":"36'119","reported_seq":378,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-01T05:49:09.782628+0000","last_change":"2024-04-01T05:49:09.782706+0000","last_active":"2024-04-01T05:49:09.782628+0000","last_peered":"2024-04-01T05:49:09.782628+0000","last_clean":"2024-04-01T05:49:09.782628+0000","last_became_active":"2024-04-01T00:13:36.812952+0000","last_became_peered":"2024-04-01T00:13:36.812952+0000","last_unstale":"2024-04-01T05:49:09.782628+0000","last_undegraded":"2024-04-01T05:49:09.782628+0000","last_fullsized":"2024-04-01T05:49:09.782628+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'119","last_scrub_stamp":"2024-04-01T05:49:09.782549+0000","last_deep_scrub":"36'119","last_deep_scrub_stamp":"2024-04-01T05:49:09.782549+0000","last_clean_scrub_stamp":"2024-04-01T05:49:09.782549+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-02T14:00:51.912402+0000","scrub_duration":270,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834578880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1635812,"kb_used_data":988,"kb_used_omap":388,"kb_used_meta":1634427,"kb_avail":92736028,"statfs":{"total":96636764160,"available":94961692672,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":398295,"internal_metadata":1673653289},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834578881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1626080,"kb_used_data":408,"kb_used_omap":406,"kb_used_meta":1625257,"kb_avail":92745760,"statfs":{"total":96636764160,"available":94971658240,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":416539,"internal_metadata":1664263397},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834578882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1789732,"kb_used_data":408,"kb_used_omap":144,"kb_used_meta":1789167,"kb_avail":92582108,"statfs":{"total":96636764160,"available":94804078592,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":147804,"internal_metadata":1832107684},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834578879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1805024,"kb_used_data":408,"kb_used_omap":839,"kb_used_meta":1803768,"kb_avail":92566816,"statfs":{"total":96636764160,"available":94788419584,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":859451,"internal_metadata":1847059141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834578841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1174048,"kb_used_data":408,"kb_used_omap":1243,"kb_used_meta":1172388,"kb_avail":93197792,"statfs":{"total":96636764160,"available":95434539008,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1273266,"internal_metadata":1200525902},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834578860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1320224,"kb_used_data":408,"kb_used_omap":315,"kb_used_meta":1319492,"kb_avail":93051616,"statfs":{"total":96636764160,"available":95284854784,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":323406,"internal_metadata":1351159986},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834578854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1694944,"kb_used_data":408,"kb_used_omap":1472,"kb_used_meta":1693055,"kb_avail":92676896,"statfs":{"total":96636764160,"available":94901141504,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507447,"internal_metadata":1733689225},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834578874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1506468,"kb_used_data":988,"kb_used_omap":282,"kb_used_meta":1505189,"kb_avail":92865372,"statfs":{"total":96636764160,"available":95094140928,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":289002,"internal_metadata":1541314326},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-01T05:49:30.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-01T05:49:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.610+0000 7f314f6eb640 1 Processor -- start 2024-04-01T05:49:30.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.610+0000 7f314f6eb640 1 -- start start 2024-04-01T05:49:30.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.612+0000 7f314f6eb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f314815e3d0 0x7f3148072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.612+0000 7f314f6eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31480afe90 0x7f314806f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.612+0000 7f314f6eb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f314806f870 0x7f314806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.612+0000 7f314f6eb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7f31480ce810 con 0x7f314815e3d0 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314f6eb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f31480d0810 con 0x7f314806f870 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314f6eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f31480cea60 con 0x7f31480afe90 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f314815e3d0 0x7f3148072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f314815e3d0 0x7f3148072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57948/0 (socket says 172.21.15.64:57948) 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 -- 172.21.15.64:0/3786503454 learned_addr learned my addr 172.21.15.64:0/3786503454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 -- 172.21.15.64:0/3786503454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31480afe90 msgr2=0x7f314806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 --2- 172.21.15.64:0/3786503454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31480afe90 0x7f314806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 -- 172.21.15.64:0/3786503454 >> v2:172.21.15.100:3300/0 conn(0x7f314806f870 msgr2=0x7f314806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 --2- 172.21.15.64:0/3786503454 >> v2:172.21.15.100:3300/0 conn(0x7f314806f870 0x7f314806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.613+0000 7f314d487640 1 -- 172.21.15.64:0/3786503454 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31480d04a0 con 0x7f314815e3d0 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.614+0000 7f314d487640 1 --2- 172.21.15.64:0/3786503454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314815e3d0 0x7f3148072670 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3148067e90 tx=0x7f314402be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c965c67acbff6dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.614+0000 7f313ffff640 1 -- 172.21.15.64:0/3786503454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314402ca10 con 0x7f314815e3d0 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.614+0000 7f313ffff640 1 -- 172.21.15.64:0/3786503454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314402cbb0 con 0x7f314815e3d0 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.614+0000 7f313ffff640 1 -- 172.21.15.64:0/3786503454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440343f0 con 0x7f314815e3d0 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.615+0000 7f314f6eb640 1 -- 172.21.15.64:0/3786503454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314815e3d0 msgr2=0x7f3148072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:30.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.615+0000 7f314f6eb640 1 --2- 172.21.15.64:0/3786503454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314815e3d0 0x7f3148072670 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3148067e90 tx=0x7f314402be70 comp rx=0 tx=0).stop 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.615+0000 7f314f6eb640 1 -- 172.21.15.64:0/3786503454 shutdown_connections 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 --2- 172.21.15.64:0/3786503454 >> v2:172.21.15.100:3300/0 conn(0x7f314806f870 0x7f314806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 --2- 172.21.15.64:0/3786503454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31480afe90 0x7f314806f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 --2- 172.21.15.64:0/3786503454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314815e3d0 0x7f3148072670 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 -- 172.21.15.64:0/3786503454 >> 172.21.15.64:0/3786503454 conn(0x7f31480d0c40 msgr2=0x7f31480d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 -- 172.21.15.64:0/3786503454 shutdown_connections 2024-04-01T05:49:30.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.616+0000 7f314f6eb640 1 -- 172.21.15.64:0/3786503454 wait complete. 2024-04-01T05:49:30.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.619+0000 7f314f6eb640 1 Processor -- start 2024-04-01T05:49:30.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.619+0000 7f314f6eb640 1 -- start start 2024-04-01T05:49:30.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 0x7f31480b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 0x7f314815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31480ceec0 con 0x7f31480afe90 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f31480ced40 con 0x7f314806f870 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314f6eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f314806b600 con 0x7f314815e3d0 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314cc86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:30.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314cc86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57956/0 (socket says 172.21.15.64:57956) 2024-04-01T05:49:30.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.621+0000 7f314cc86640 1 -- 172.21.15.64:0/135375066 learned_addr learned my addr 172.21.15.64:0/135375066 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:30.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314d487640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 0x7f31480b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:30.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314dc88640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 0x7f314815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:30.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314cc86640 1 -- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 msgr2=0x7f314815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:30.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314cc86640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 0x7f314815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314cc86640 1 -- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 msgr2=0x7f31480b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:30.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314cc86640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 0x7f31480b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:30.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314cc86640 1 -- 172.21.15.64:0/135375066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31480c4470 con 0x7f31480afe90 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314dc88640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 0x7f314815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.622+0000 7f314d487640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 0x7f31480b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.623+0000 7f314cc86640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f313800b4a0 tx=0x7f313800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.623+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138014070 con 0x7f31480afe90 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.623+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138010610 con 0x7f31480afe90 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.623+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31480c4750 con 0x7f31480afe90 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.623+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138010a00 con 0x7f31480afe90 2024-04-01T05:49:30.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.624+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31480c4d10 con 0x7f31480afe90 2024-04-01T05:49:30.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.626+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31480688b0 con 0x7f31480afe90 2024-04-01T05:49:30.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.627+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7f3138017070 con 0x7f31480afe90 2024-04-01T05:49:30.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.629+0000 7f313dffb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 0x7f312c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:30.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.629+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f3138089e80 con 0x7f31480afe90 2024-04-01T05:49:30.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.629+0000 7f314d487640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 0x7f312c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:30.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.630+0000 7f314d487640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 0x7f312c06de50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3144035000 tx=0x7f3144030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:30.642+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7f31380587d0 con 0x7f31480afe90 2024-04-01T05:49:31.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.174+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f31481615c0 con 0x7f31480afe90 2024-04-01T05:49:31.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.178+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f313805c870 con 0x7f31480afe90 2024-04-01T05:49:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.179+0000 7f313dffb640 1 -- 172.21.15.64:0/135375066 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f31380610a0 con 0x7f31480afe90 2024-04-01T05:49:31.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.184+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 msgr2=0x7f312c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 0x7f312c06de50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3144035000 tx=0x7f3144030000 comp rx=0 tx=0).stop 2024-04-01T05:49:31.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 msgr2=0x7f31480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f313800b4a0 tx=0x7f313800b960 comp rx=0 tx=0).stop 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 shutdown_connections 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7f312c06b990 0x7f312c06de50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f314815e3d0 0x7f314815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31480afe90 0x7f31480b4e50 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 --2- 172.21.15.64:0/135375066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f314806f870 0x7f31480b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 >> 172.21.15.64:0/135375066 conn(0x7f31480d0c40 msgr2=0x7f31480cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:31.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 shutdown_connections 2024-04-01T05:49:31.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.185+0000 7f314f6eb640 1 -- 172.21.15.64:0/135375066 wait complete. 2024-04-01T05:49:31.210 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-01T05:49:31.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-01T05:49:31.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.484+0000 7ff65b6a4640 1 Processor -- start 2024-04-01T05:49:31.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.484+0000 7ff65b6a4640 1 -- start start 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff65b6a4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff65415c010 0x7ff6540d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff65b6a4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6540d09f0 0x7ff654160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff65b6a4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6540d2030 con 0x7ff6540cc810 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff65b6a4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 -- 0x7ff654160ac0 con 0x7ff6540d09f0 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff65b6a4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff6540d2310 con 0x7ff65415c010 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff659c41640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6540d09f0 0x7ff654160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff659440640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff65415c010 0x7ff6540d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff658c3f640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff6540cc810 0x7ff6540d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.64:40722/0 (socket says 172.21.15.64:40722) 2024-04-01T05:49:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.486+0000 7ff658c3f640 1 -- 172.21.15.64:0/1933379059 learned_addr learned my addr 172.21.15.64:0/1933379059 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-04-01T05:49:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.487+0000 7ff659c41640 1 -- 172.21.15.64:0/1933379059 >> v2:172.21.15.100:3300/0 conn(0x7ff65415c010 msgr2=0x7ff6540d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-01T05:49:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.487+0000 7ff659c41640 1 --2- 172.21.15.64:0/1933379059 >> v2:172.21.15.100:3300/0 conn(0x7ff65415c010 0x7ff6540d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.487+0000 7ff659c41640 1 -- 172.21.15.64:0/1933379059 >> v1:172.21.15.100:6789/0 conn(0x7ff6540cc810 legacy=0x7ff6540d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.487+0000 7ff647fff640 1 -- 172.21.15.64:0/1933379059 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123410244 0 0) 0x7ff6540d2030 con 0x7ff6540cc810 2024-04-01T05:49:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.487+0000 7ff659c41640 1 -- 172.21.15.64:0/1933379059 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6540d25f0 con 0x7ff6540d09f0 2024-04-01T05:49:31.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.489+0000 7ff659c41640 1 --2- 172.21.15.64:0/1933379059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6540d09f0 0x7ff654160580 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff6540b4890 tx=0x7ff64802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a8da0a1663814d server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:31.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.489+0000 7ff647fff640 1 -- 172.21.15.64:0/1933379059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648035020 con 0x7ff6540d09f0 2024-04-01T05:49:31.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.489+0000 7ff647fff640 1 -- 172.21.15.64:0/1933379059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff648031830 con 0x7ff6540d09f0 2024-04-01T05:49:31.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.489+0000 7ff647fff640 1 -- 172.21.15.64:0/1933379059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648031c20 con 0x7ff6540d09f0 2024-04-01T05:49:31.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.490+0000 7ff65b6a4640 1 -- 172.21.15.64:0/1933379059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6540d09f0 msgr2=0x7ff654160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.490+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/1933379059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6540d09f0 0x7ff654160580 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff6540b4890 tx=0x7ff64802b8d0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 -- 172.21.15.64:0/1933379059 shutdown_connections 2024-04-01T05:49:31.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/1933379059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6540d09f0 0x7ff654160580 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/1933379059 >> v2:172.21.15.100:3300/0 conn(0x7ff65415c010 0x7ff6540d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 -- 172.21.15.64:0/1933379059 >> 172.21.15.64:0/1933379059 conn(0x7ff65409a980 msgr2=0x7ff65409ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:31.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 -- 172.21.15.64:0/1933379059 shutdown_connections 2024-04-01T05:49:31.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.491+0000 7ff65b6a4640 1 -- 172.21.15.64:0/1933379059 wait complete. 2024-04-01T05:49:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.494+0000 7ff65b6a4640 1 Processor -- start 2024-04-01T05:49:31.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.494+0000 7ff65b6a4640 1 -- start start 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 0x7ff6541eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6540d09f0 0x7ff6541f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6540b1e60 con 0x7ff65415c010 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff65415c3f0 con 0x7ff6540cc810 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff65b6a4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff6540b1fe0 con 0x7ff6540d09f0 2024-04-01T05:49:31.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff659c41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff659c41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57968/0 (socket says 172.21.15.64:57968) 2024-04-01T05:49:31.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.496+0000 7ff659c41640 1 -- 172.21.15.64:0/477494379 learned_addr learned my addr 172.21.15.64:0/477494379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-01T05:49:31.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff658c3f640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6540d09f0 0x7ff6541f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659440640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 0x7ff6541eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 -- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6540d09f0 msgr2=0x7ff6541f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6540d09f0 0x7ff6541f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 -- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 msgr2=0x7ff6541eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 0x7ff6541eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 -- 172.21.15.64:0/477494379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6541f2570 con 0x7ff65415c010 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659440640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 0x7ff6541eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.497+0000 7ff659c41640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff64802bee0 tx=0x7ff6480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.498+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648041070 con 0x7ff65415c010 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.498+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff64803b860 con 0x7ff65415c010 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.498+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648035040 con 0x7ff65415c010 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.498+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6541f2850 con 0x7ff65415c010 2024-04-01T05:49:31.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.498+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6540c6d70 con 0x7ff65415c010 2024-04-01T05:49:31.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.499+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6540c6f70 con 0x7ff65415c010 2024-04-01T05:49:31.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.501+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 90156+0+0 (secure 0 0 0) 0x7ff648038020 con 0x7ff65415c010 2024-04-01T05:49:31.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.502+0000 7ff645ffb640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 0x7ff63806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-01T05:49:31.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.502+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff6480abff0 con 0x7ff65415c010 2024-04-01T05:49:31.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.503+0000 7ff659440640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 0x7ff63806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-01T05:49:31.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.503+0000 7ff659440640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 0x7ff63806de50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff65406f5c0 tx=0x7ff64c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-01T05:49:31.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.516+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+183808 (secure 0 0 0) 0x7ff6540c6f70 con 0x7ff65415c010 2024-04-01T05:49:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.781+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6540688b0 con 0x7ff65415c010 2024-04-01T05:49:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.783+0000 7ff645ffb640 1 -- 172.21.15.64:0/477494379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3869 (secure 0 0 0) 0x7ff64807ad70 con 0x7ff65415c010 2024-04-01T05:49:31.784 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-01T05:49:31.784 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/543747286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":543747286},{"type":"v1","addr":"172.21.15.64:6834","nonce":543747286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3447709368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3447709368},{"type":"v1","addr":"172.21.15.64:6835","nonce":3447709368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1225022682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1225022682},{"type":"v1","addr":"172.21.15.100:6836","nonce":1225022682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":9835,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/629004134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":629004134},{"type":"v1","addr":"172.21.15.100:6837","nonce":629004134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-04-01T05:49:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 msgr2=0x7ff63806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 0x7ff63806de50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff65406f5c0 tx=0x7ff64c008040 comp rx=0 tx=0).stop 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 msgr2=0x7ff6541f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff64802bee0 tx=0x7ff6480046c0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 shutdown_connections 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:6832/93206356,v1:172.21.15.100:6833/93206356] conn(0x7ff63806b990 0x7ff63806de50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65415c010 0x7ff6541f1f80 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6540d09f0 0x7ff6541f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 --2- 172.21.15.64:0/477494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540cc810 0x7ff6541eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-01T05:49:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 >> 172.21.15.64:0/477494379 conn(0x7ff65409a980 msgr2=0x7ff6540d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-01T05:49:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 shutdown_connections 2024-04-01T05:49:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-01T05:49:31.785+0000 7ff65b6a4640 1 -- 172.21.15.64:0/477494379 wait complete. 2024-04-01T05:49:31.800 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-01T05:49:31.800 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-01T05:49:31.801 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-01T05:49:31.801 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:31.821 INFO:tasks.ceph.mds.a:Stopped 2024-04-01T05:49:31.821 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-01T05:49:31.821 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:31.881 INFO:tasks.ceph.mds.c:Stopped 2024-04-01T05:49:31.882 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-01T05:49:31.882 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:31.894 INFO:tasks.ceph.mds.b:Stopped 2024-04-01T05:49:31.894 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-01T05:49:31.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:31.947 INFO:tasks.ceph.mds.d:Stopped 2024-04-01T05:49:31.947 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-01T05:49:31.947 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-01T05:49:31.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:36.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-01T05:49:38.050 INFO:tasks.ceph.osd.0:Stopped 2024-04-01T05:49:38.050 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-01T05:49:38.050 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:42.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-01T05:49:42.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-01T05:49:44.152 INFO:tasks.ceph.osd.1:Stopped 2024-04-01T05:49:44.153 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-01T05:49:44.153 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:48.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-01T05:49:48.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-01T05:49:50.254 INFO:tasks.ceph.osd.2:Stopped 2024-04-01T05:49:50.255 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-01T05:49:50.255 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:49:54.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-01T05:49:54.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-01T05:49:56.356 INFO:tasks.ceph.osd.3:Stopped 2024-04-01T05:49:56.357 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-01T05:49:56.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:00.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-01T05:50:00.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-01T05:50:02.458 INFO:tasks.ceph.osd.4:Stopped 2024-04-01T05:50:02.459 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-01T05:50:02.459 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:06.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-01T05:50:06.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-01T05:50:08.561 INFO:tasks.ceph.osd.5:Stopped 2024-04-01T05:50:08.562 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-01T05:50:08.562 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:11.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-01T05:50:11.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-01T05:50:14.664 INFO:tasks.ceph.osd.6:Stopped 2024-04-01T05:50:14.664 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-01T05:50:14.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.763 INFO:tasks.ceph.osd.7:Stopped 2024-04-01T05:50:14.763 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-01T05:50:14.763 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-01T05:50:14.764 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.815 INFO:tasks.ceph.mgr.y:Stopped 2024-04-01T05:50:14.815 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-01T05:50:14.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.845 INFO:tasks.ceph.mgr.x:Stopped 2024-04-01T05:50:14.846 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-01T05:50:14.846 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-01T05:50:14.846 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.898 INFO:tasks.ceph.mon.a:Stopped 2024-04-01T05:50:14.899 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-01T05:50:14.899 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.916 INFO:tasks.ceph.mon.b:Stopped 2024-04-01T05:50:14.917 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-01T05:50:14.917 INFO:teuthology.orchestra.run:waiting for 300 2024-04-01T05:50:14.974 INFO:tasks.ceph.mon.c:Stopped 2024-04-01T05:50:14.974 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-01T05:50:14.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-04-01T05:50:15.107 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:50:15.107 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-01T05:50:15.366 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:50:15.366 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-01T05:50:15.562 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:50:15.563 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-01T05:50:15.744 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T05:50:15.744 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-01T05:50:15.963 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:50:15.966 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-01T05:50:16.131 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:50:16.132 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-01T05:50:16.300 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:50:16.300 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-01T05:50:16.382 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T05:50:16.382 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-01T05:50:16.488 INFO:tasks.ceph:Archiving mon data... 2024-04-01T05:50:16.489 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/data/mon.a.tgz 2024-04-01T05:50:16.489 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-04-01T05:50:16.523 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.xEOvYuxZiM 2024-04-01T05:50:16.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xEOvYuxZiM 2024-04-01T05:50:16.862 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-01T05:50:16.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.xEOvYuxZiM 2024-04-01T05:50:17.019 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.xEOvYuxZiM is 564KB 2024-04-01T05:50:17.113 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.xEOvYuxZiM 2024-04-01T05:50:17.145 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/data/mon.b.tgz 2024-04-01T05:50:17.145 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-01T05:50:17.161 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.gQmRs5ZKp2 2024-04-01T05:50:17.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gQmRs5ZKp2 2024-04-01T05:50:17.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.gQmRs5ZKp2 2024-04-01T05:50:17.431 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.gQmRs5ZKp2 is 567KB 2024-04-01T05:50:17.530 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.gQmRs5ZKp2 2024-04-01T05:50:17.544 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/data/mon.c.tgz 2024-04-01T05:50:17.545 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-01T05:50:17.598 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.Pk21NoZb51 2024-04-01T05:50:17.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Pk21NoZb51 2024-04-01T05:50:17.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.Pk21NoZb51 2024-04-01T05:50:17.833 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.Pk21NoZb51 is 567KB 2024-04-01T05:50:17.931 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.Pk21NoZb51 2024-04-01T05:50:17.945 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-01T05:50:17.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-01T05:50:17.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-01T05:50:18.075 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-01T05:50:18.090 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-01T05:50:18.090 INFO:tasks.ceph:Archiving crash dumps... 2024-04-01T05:50:18.092 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi064/crash 2024-04-01T05:50:18.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-01T05:50:18.147 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi100/crash 2024-04-01T05:50:18.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-01T05:50:18.174 INFO:tasks.ceph:Shutting down logrotate 2024-04-01T05:50:18.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:50:18.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-01T05:50:18.247 INFO:teuthology.orchestra.run.smithi064.stderr:ceph-osd: no process found 2024-04-01T05:56:01.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-01T05:56:01.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-01T05:56:01.882 INFO:tasks.ceph:Compressing logs... 2024-04-01T05:56:01.883 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-01T05:56:01.886 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-01T05:56:01.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.517782.log 2024-04-01T05:56:01.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212392.log 2024-04-01T05:56:01.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.517782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222446.log 2024-04-01T05:56:01.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164937.log 2024-04-01T05:56:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.496528.log 2024-04-01T05:56:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.542749.log 2024-04-01T05:56:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411828.log 2024-04-01T05:56:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486619.log 2024-04-01T05:56:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.237038.log 2024-04-01T05:56:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193390.log 2024-04-01T05:56:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243164.log 2024-04-01T05:56:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548928.log 2024-04-01T05:56:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487105.log 2024-04-01T05:56:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220592.log 2024-04-01T05:56:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.237883.log 2024-04-01T05:56:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-04-01T05:56:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.272526.log 2024-04-01T05:56:02.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2024-04-01T05:56:02.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.502521.log 2024-04-01T05:56:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383590.log 2024-04-01T05:56:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74053.log 2024-04-01T05:56:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.497381.log 2024-04-01T05:56:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435183.log 2024-04-01T05:56:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201095.log 2024-04-01T05:56:02.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.496528.log: /var/log/ceph/ceph-client.admin.435183.log: /var/log/ceph/ceph-client.admin.542749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242331.log 2024-04-01T05:56:02.009 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.486619.log: /var/log/ceph/ceph-client.admin.49708.log: /var/log/ceph/ceph-client.admin.548928.log: /var/log/ceph/ceph-client.admin.51816.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.433472.log 2024-04-01T05:56:02.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.74053.log: /var/log/ceph/ceph-client.admin.487105.log: /var/log/ceph/ceph-client.admin.502521.log: /var/log/ceph/ceph-client.admin.497381.log: /var/log/ceph/ceph-client.admin.164937.log: /var/log/ceph/ceph-client.admin.212392.log: /var/log/ceph/ceph-client.admin.222446.log: /var/log/ceph/ceph-client.admin.243164.log: /var/log/ceph/ceph-client.admin.237038.log: /var/log/ceph/ceph-client.admin.193390.log: /var/log/ceph/ceph-client.admin.220592.log: /var/log/ceph/ceph-client.admin.237883.log: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.542749.log.gz 2024-04-01T05:56:02.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.272526.log: gzip/var/log/ceph/ceph-client.admin.411828.log: /var/log/ceph/ceph-client.admin.383590.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.268663.log 2024-04-01T05:56:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.242331.log: /var/log/ceph/ceph-client.admin.201095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350260.log 2024-04-01T05:56:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.433472.log: /var/log/ceph/ceph-client.admin.268663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113672.log 2024-04-01T05:56:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49466.log/var/log/ceph/ceph-client.admin.350260.log: 2024-04-01T05:56:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432629.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113672.log: /var/log/ceph/ceph-client.admin.49466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47999.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.432629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198395.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423837.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47999.log: /var/log/ceph/ceph-client.admin.198395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.266143.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.423837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144056.log 2024-04-01T05:56:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469934.log 2024-04-01T05:56:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.266143.log: /var/log/ceph/ceph-client.admin.144056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-04-01T05:56:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.469934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232758.log 2024-04-01T05:56:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-01T05:56:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.37401.log: /var/log/ceph/ceph-client.admin.232758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180931.log 2024-04-01T05:56:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202454.log 2024-04-01T05:56:02.033 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133726.log 2024-04-01T05:56:02.033 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.180931.log: /var/log/ceph/ceph-client.admin.202454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565499.log 2024-04-01T05:56:02.033 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.133726.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.438603.log 2024-04-01T05:56:02.033 INFO:teuthology.orchestra.run.smithi064.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.517782.log.gz 2024-04-01T05:56:02.034 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179225.log 2024-04-01T05:56:02.034 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565499.log: /var/log/ceph/ceph-client.admin.438603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.496519.log 2024-04-01T05:56:02.034 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.565499.log.gz 2024-04-01T05:56:02.034 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.520293.log 2024-04-01T05:56:02.034 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.496519.log: /var/log/ceph/ceph-client.admin.179225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356411.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.520293.log: /var/log/ceph/ceph-client.admin.356411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164496.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.51392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.539642.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.164496.log: /var/log/ceph/ceph-client.admin.37653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467742.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.539642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370004.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47553.log 2024-04-01T05:56:02.035 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.467742.log: /var/log/ceph/ceph-client.admin.370004.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.539642.log.gz 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74053.log.gz 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.521728.log 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462439.log 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233615.log 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.521728.log: /var/log/ceph/ceph-client.admin.462439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353231.log 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.233615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.528804.log 2024-04-01T05:56:02.036 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205598.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.353231.log: /var/log/ceph/ceph-client.admin.528804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564333.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.205598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448022.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164066.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.448022.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.366188.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.564333.log.gz 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.530500.log 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.366188.log: /var/log/ceph/ceph-client.admin.164066.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47999.log.gz 2024-04-01T05:56:02.037 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.502952.log 2024-04-01T05:56:02.038 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442029.log 2024-04-01T05:56:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44055.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.44055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37676.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37676.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43599.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.43599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-04-01T05:56:02.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37756.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37596.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-01T05:56:02.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37476.log 2024-04-01T05:56:02.055 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-01T05:56:02.055 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37476.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-01T05:56:02.055 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-04-01T05:56:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37636.log 2024-04-01T05:56:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37556.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-01T05:56:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37636.log: /var/log/ceph/ceph-mon.c.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-04-01T05:56:02.056 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37676.log.gz 2024-04-01T05:56:02.057 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37636.log.gz 2024-04-01T05:56:02.057 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-04-01T05:56:02.057 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-04-01T05:56:02.062 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-04-01T05:56:02.079 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.3.44055.log.gz 2024-04-01T05:56:02.084 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.530500.log: /var/log/ceph/ceph-client.admin.502952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44519.log 2024-04-01T05:56:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.442029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421388.log 2024-04-01T05:56:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.547639.log 2024-04-01T05:56:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.421388.log: /var/log/ceph/ceph-client.admin.44519.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49466.log.gz 2024-04-01T05:56:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383599.log 2024-04-01T05:56:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.547639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.263617.log 2024-04-01T05:56:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.522979.log 2024-04-01T05:56:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.383599.log: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.547639.log.gz 2024-04-01T05:56:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.263617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.263095.log 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.522979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477801.log 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.263095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.477801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440310.log 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449746.log 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50936.log: /var/log/ceph/ceph-client.admin.440310.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.502521.log.gz 2024-04-01T05:56:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442871.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193399.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.449746.log: /var/log/ceph/ceph-client.admin.442871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.530491.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193852.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.530491.log: /var/log/ceph/ceph-client.admin.193399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.259657.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.193852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244051.log 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.435183.log.gz 2024-04-01T05:56:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.259657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483910.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.244051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192127.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.556909.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.483910.log: /var/log/ceph/ceph-client.admin.192127.log: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.383590.log.gz 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441566.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.556909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147527.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406505.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.441566.log: /var/log/ceph/ceph-client.admin.147527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.495679.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384862.log 2024-04-01T05:56:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.406505.log: /var/log/ceph/ceph-client.admin.495679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.393285.log 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.384862.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.432629.log.gz 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.554680.log 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212806.log 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.393285.log: /var/log/ceph/ceph-client.admin.554680.log: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.554680.log.gz 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.433472.log.gz 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450181.log 2024-04-01T05:56:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.212806.log: /var/log/ceph/ceph-client.admin.468632.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.450181.log: --verbose/var/log/ceph/ceph-client.admin.468632.log: -- /var/log/ceph/ceph-client.admin.464124.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.350260.log.gz 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.561362.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136265.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.464124.log: /var/log/ceph/ceph-client.admin.561362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115960.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.136265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-04-01T05:56:02.091 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.556909.log.gz 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433481.log 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115960.log: /var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157000.log 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.561362.log.gz 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.433481.log: --verbose -- /var/log/ceph/ceph-client.admin.551132.log 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.157000.log: /var/log/ceph/ceph-client.admin.551132.log: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.551132.log.gz 2024-04-01T05:56:02.092 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.423837.log.gz 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343903.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.45201.log: --verbose -- /var/log/ceph/ceph-client.admin.162239.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206487.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.343903.log: /var/log/ceph/ceph-client.admin.162239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411325.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410459.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.411325.log: /var/log/ceph/ceph-client.admin.206487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214536.log 2024-04-01T05:56:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.508742.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.410459.log: /var/log/ceph/ceph-client.admin.214536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244042.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.508742.log: -- /var/log/ceph/ceph-client.admin.111664.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213650.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.244042.log: /var/log/ceph/ceph-client.admin.111664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466895.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.213650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281697.log 2024-04-01T05:56:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.466895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463703.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.281697.log: --verbose -- /var/log/ceph/ceph-client.admin.533451.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351112.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.463703.log: /var/log/ceph/ceph-client.admin.533451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44119.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381890.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44119.log: /var/log/ceph/ceph-client.admin.351112.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384011.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222025.log 2024-04-01T05:56:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.384011.log: /var/log/ceph/ceph-client.admin.381890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183057.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439038.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.183057.log: /var/log/ceph/ceph-client.admin.222025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145342.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.439038.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.142276.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564207.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.145342.log: /var/log/ceph/ceph-client.admin.142276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.252351.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.498222.log 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.252351.log: /var/log/ceph/ceph-client.admin.564207.log: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.548928.log.gz 2024-04-01T05:56:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439875.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.498222.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.564207.log.gz 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129000.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.439875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343490.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.129000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.539178.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.343490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230608.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.539178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195719.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155636.log 2024-04-01T05:56:02.097 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.539178.log.gz 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.230608.log: /var/log/ceph/ceph-client.admin.195719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48167.log 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50684.log 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.155636.log: /var/log/ceph/ceph-client.admin.48167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473026.log 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486197.log 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.473026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200644.log 2024-04-01T05:56:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.486197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450172.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.200644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395675.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.450172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.207754.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.553793.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.395675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128122.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.207754.log: /var/log/ceph/ceph-client.admin.553793.log: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.553793.log.gz 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367466.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422269.log 2024-04-01T05:56:02.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.367466.log: /var/log/ceph/ceph-client.admin.128122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120721.log 2024-04-01T05:56:02.100 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.193390.log.gz 2024-04-01T05:56:02.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.422269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381452.log 2024-04-01T05:56:02.100 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.201095.log.gz 2024-04-01T05:56:02.100 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.120721.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.564946.log 2024-04-01T05:56:02.101 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-01T05:56:02.126 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.1.43599.log.gz 2024-04-01T05:56:02.130 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541004.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564946.log: /var/log/ceph/ceph-client.admin.381452.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.564946.log.gz 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.248669.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.541004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489986.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.541004.log.gz 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.248669.log: /var/log/ceph/ceph-client.admin.44897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282970.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.555573.log 2024-04-01T05:56:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.543928.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.282970.log: /var/log/ceph/ceph-client.admin.555573.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.555573.log.gz 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.222446.log.gz 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257541.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.543928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448031.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.257541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278096.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.448031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356823.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427912.log 2024-04-01T05:56:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.278096.log: /var/log/ceph/ceph-client.admin.356823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135836.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.259235.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.135836.log: /var/log/ceph/ceph-client.admin.427912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565079.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.526694.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565079.log: /var/log/ceph/ceph-client.admin.259235.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.565079.log.gz 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565541.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.441566.log.gz 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.526694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141391.log 2024-04-01T05:56:02.133 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.442029.log.gz 2024-04-01T05:56:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.505107.log 2024-04-01T05:56:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565541.log: /var/log/ceph/ceph-client.admin.141391.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.440310.log.gz 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.565541.log.gz 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.530077.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.505107.log: /var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449304.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257552.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.449304.log: /var/log/ceph/ceph-client.admin.530077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218044.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429214.log 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.257552.log: /var/log/ceph/ceph-client.admin.218044.log: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.462439.log.gz 2024-04-01T05:56:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231072.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484770.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.429214.log: /var/log/ceph/ceph-client.admin.231072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233170.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146661.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.233170.log: /var/log/ceph/ceph-client.admin.484770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.280386.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.146661.log: --verbose -- /var/log/ceph/ceph-client.admin.391270.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 79.2% -- replaced with /var/log/ceph/ceph-client.admin.467742.log.gz 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.44375.log 2024-04-01T05:56:02.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.391270.log: /var/log/ceph/ceph-client.admin.280386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156063.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140488.log/var/log/ceph/ceph-client.admin.156063.log: 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209859.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.140488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.393276.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.209859.log: gzip 77.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.449755.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.243164.log.gz 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.393276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345182.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480563.log 2024-04-01T05:56:02.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.449755.log: /var/log/ceph/ceph-client.admin.345182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.207333.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346451.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.207333.log: /var/log/ceph/ceph-client.admin.480563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113162.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.279886.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113162.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.343490.log.gz 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.346451.log: gzip -5 80.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.268663.log.gz 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.52002.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.279886.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.343481.log 2024-04-01T05:56:02.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234881.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.164066.log.gz 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.343481.log: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.164937.log.gz 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361906.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484348.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.234881.log: /var/log/ceph/ceph-client.admin.361906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374232.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.484348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217633.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451765.log 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.374232.log: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.205598.log.gz 2024-04-01T05:56:02.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.217633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427486.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.451765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441575.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436914.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.427486.log: /var/log/ceph/ceph-client.admin.441575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210280.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.545025.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.436914.log: /var/log/ceph/ceph-client.admin.210280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154354.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125636.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.545025.log: /var/log/ceph/ceph-client.admin.154354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74011.log 2024-04-01T05:56:02.140 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.496519.log.gz 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.125636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.497381.log.gz 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113153.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184466.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199304.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49256.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.392785.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.426192.log 2024-04-01T05:56:02.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.249081.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129421.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111673.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349405.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142285.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.505674.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46031.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407409.log 2024-04-01T05:56:02.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.265301.log 2024-04-01T05:56:02.143 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132454.log 2024-04-01T05:56:02.143 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.247812.log 2024-04-01T05:56:02.144 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-04-01T05:56:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417933.log 2024-04-01T05:56:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132032.log 2024-04-01T05:56:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565877.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.265292.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199745.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431375.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.279397.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423943.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488107.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482024.log 2024-04-01T05:56:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242752.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.390282.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.526263.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481050.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281275.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.500819.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48563.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217203.log 2024-04-01T05:56:02.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358550.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.542326.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349414.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437326.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408706.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441154.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423789.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215877.log 2024-04-01T05:56:02.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179661.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379231.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379677.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430070.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359394.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.507038.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50726.log 2024-04-01T05:56:02.156 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282536.log 2024-04-01T05:56:02.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-04-01T05:56:02.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478277.log 2024-04-01T05:56:02.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153471.log 2024-04-01T05:56:02.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427035.log 2024-04-01T05:56:02.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282961.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.525850.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139638.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231906.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436463.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.262644.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152614.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381000.log 2024-04-01T05:56:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134993.log 2024-04-01T05:56:02.163 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-04-01T05:56:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434740.log 2024-04-01T05:56:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-04-01T05:56:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154784.log 2024-04-01T05:56:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353667.log 2024-04-01T05:56:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49214.log 2024-04-01T05:56:02.165 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477371.log 2024-04-01T05:56:02.165 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219315.log 2024-04-01T05:56:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47915.log 2024-04-01T05:56:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.251517.log 2024-04-01T05:56:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190273.log 2024-04-01T05:56:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.509597.log 2024-04-01T05:56:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.504275.log 2024-04-01T05:56:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478707.log 2024-04-01T05:56:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231063.log 2024-04-01T05:56:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112712.log 2024-04-01T05:56:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464558.log 2024-04-01T05:56:02.170 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357244.log 2024-04-01T05:56:02.171 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180088.log 2024-04-01T05:56:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.517791.log 2024-04-01T05:56:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406924.log 2024-04-01T05:56:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.495258.log 2024-04-01T05:56:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.519009.log 2024-04-01T05:56:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127710.log 2024-04-01T05:56:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217212.log 2024-04-01T05:56:02.175 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.503855.log 2024-04-01T05:56:02.175 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.280395.log 2024-04-01T05:56:02.177 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51434.log 2024-04-01T05:56:02.177 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-04-01T05:56:02.178 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44981.log 2024-04-01T05:56:02.178 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160324.log 2024-04-01T05:56:02.179 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370417.log 2024-04-01T05:56:02.182 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565794.log 2024-04-01T05:56:02.182 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485225.log 2024-04-01T05:56:02.182 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.509588.log 2024-04-01T05:56:02.182 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-04-01T05:56:02.183 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351523.log 2024-04-01T05:56:02.183 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346872.log 2024-04-01T05:56:02.184 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.532186.log 2024-04-01T05:56:02.184 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469095.log 2024-04-01T05:56:02.184 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49808.log 2024-04-01T05:56:02.185 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2024-04-01T05:56:02.185 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363588.log 2024-04-01T05:56:02.186 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463291.log 2024-04-01T05:56:02.186 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356402.log 2024-04-01T05:56:02.187 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565205.log 2024-04-01T05:56:02.187 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536960.log 2024-04-01T05:56:02.188 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410901.log 2024-04-01T05:56:02.188 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413206.log 2024-04-01T05:56:02.189 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476538.log 2024-04-01T05:56:02.189 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.271260.log 2024-04-01T05:56:02.190 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134151.log 2024-04-01T05:56:02.190 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.549790.log 2024-04-01T05:56:02.191 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2024-04-01T05:56:02.191 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-04-01T05:56:02.192 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188904.log 2024-04-01T05:56:02.193 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147954.log 2024-04-01T05:56:02.193 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161710.log 2024-04-01T05:56:02.194 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226727.log 2024-04-01T05:56:02.194 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.557786.log 2024-04-01T05:56:02.195 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424167.log 2024-04-01T05:56:02.195 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37476.log.gz 2024-04-01T05:56:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48209.log 2024-04-01T05:56:02.197 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423934.log 2024-04-01T05:56:02.197 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462027.log 2024-04-01T05:56:02.198 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377966.log 2024-04-01T05:56:02.198 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48926.log 2024-04-01T05:56:02.198 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361915.log 2024-04-01T05:56:02.199 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184457.log 2024-04-01T05:56:02.199 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2024-04-01T05:56:02.200 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478698.log 2024-04-01T05:56:02.200 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118557.log 2024-04-01T05:56:02.201 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.252782.log 2024-04-01T05:56:02.201 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160741.log 2024-04-01T05:56:02.202 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469086.log 2024-04-01T05:56:02.202 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50894.log 2024-04-01T05:56:02.203 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442450.log 2024-04-01T05:56:02.204 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198404.log 2024-04-01T05:56:02.204 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115073.log 2024-04-01T05:56:02.205 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241331.log 2024-04-01T05:56:02.206 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2024-04-01T05:56:02.206 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139223.log 2024-04-01T05:56:02.207 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138376.log 2024-04-01T05:56:02.207 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414498.log 2024-04-01T05:56:02.208 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355058.log 2024-04-01T05:56:02.209 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257107.log 2024-04-01T05:56:02.209 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.531765.log 2024-04-01T05:56:02.210 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.251067.log 2024-04-01T05:56:02.210 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434749.log 2024-04-01T05:56:02.211 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417520.log 2024-04-01T05:56:02.211 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.251939.log 2024-04-01T05:56:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154793.log 2024-04-01T05:56:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.488721.log 2024-04-01T05:56:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481537.log 2024-04-01T05:56:02.214 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182646.log 2024-04-01T05:56:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427044.log 2024-04-01T05:56:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206053.log 2024-04-01T05:56:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564627.log 2024-04-01T05:56:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214977.log 2024-04-01T05:56:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364018.log 2024-04-01T05:56:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241766.log 2024-04-01T05:56:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451886.log 2024-04-01T05:56:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186705.log 2024-04-01T05:56:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438612.log 2024-04-01T05:56:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.271691.log 2024-04-01T05:56:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52308.log 2024-04-01T05:56:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-04-01T05:56:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.558258.log 2024-04-01T05:56:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-04-01T05:56:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388047.log 2024-04-01T05:56:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.271269.log 2024-04-01T05:56:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564585.log 2024-04-01T05:56:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354185.log 2024-04-01T05:56:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232327.log 2024-04-01T05:56:02.224 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50390.log 2024-04-01T05:56:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.533442.log 2024-04-01T05:56:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.557795.log 2024-04-01T05:56:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.497793.log 2024-04-01T05:56:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344756.log 2024-04-01T05:56:02.227 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.238736.log 2024-04-01T05:56:02.227 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415354.log 2024-04-01T05:56:02.228 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.524586.log 2024-04-01T05:56:02.229 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431796.log 2024-04-01T05:56:02.229 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480041.log 2024-04-01T05:56:02.230 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135827.log 2024-04-01T05:56:02.230 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50054.log 2024-04-01T05:56:02.231 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541899.log 2024-04-01T05:56:02.231 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.546732.log 2024-04-01T05:56:02.232 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.532608.log 2024-04-01T05:56:02.233 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-04-01T05:56:02.233 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371285.log 2024-04-01T05:56:02.234 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.535186.log 2024-04-01T05:56:02.235 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440301.log 2024-04-01T05:56:02.235 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.549379.log 2024-04-01T05:56:02.236 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.342643.log 2024-04-01T05:56:02.236 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223772.log 2024-04-01T05:56:02.237 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146197.log 2024-04-01T05:56:02.237 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195304.log 2024-04-01T05:56:02.238 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475229.log 2024-04-01T05:56:02.238 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190809.log 2024-04-01T05:56:02.239 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351952.log 2024-04-01T05:56:02.240 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430061.log 2024-04-01T05:56:02.240 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346863.log 2024-04-01T05:56:02.241 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116447.log 2024-04-01T05:56:02.241 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.508751.log 2024-04-01T05:56:02.242 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470360.log 2024-04-01T05:56:02.243 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353222.log 2024-04-01T05:56:02.243 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349834.log 2024-04-01T05:56:02.244 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.266976.log 2024-04-01T05:56:02.244 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.543674.log 2024-04-01T05:56:02.245 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348136.log 2024-04-01T05:56:02.245 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415782.log 2024-04-01T05:56:02.246 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218464.log 2024-04-01T05:56:02.246 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204705.log 2024-04-01T05:56:02.247 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209429.log 2024-04-01T05:56:02.247 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376076.log 2024-04-01T05:56:02.248 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430488.log 2024-04-01T05:56:02.249 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231484.log 2024-04-01T05:56:02.250 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113663.log 2024-04-01T05:56:02.250 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46325.log 2024-04-01T05:56:02.250 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428785.log 2024-04-01T05:56:02.251 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.549370.log 2024-04-01T05:56:02.251 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409149.log 2024-04-01T05:56:02.252 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.504266.log 2024-04-01T05:56:02.253 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564249.log 2024-04-01T05:56:02.253 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.539651.log 2024-04-01T05:56:02.254 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.531344.log 2024-04-01T05:56:02.255 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244499.log 2024-04-01T05:56:02.255 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153055.log 2024-04-01T05:56:02.256 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130254.log 2024-04-01T05:56:02.256 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484761.log 2024-04-01T05:56:02.257 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137523.log 2024-04-01T05:56:02.258 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45198.log 2024-04-01T05:56:02.258 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218053.log 2024-04-01T05:56:02.258 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510918.log 2024-04-01T05:56:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127260.log 2024-04-01T05:56:02.260 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.447189.log 2024-04-01T05:56:02.260 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345603.log 2024-04-01T05:56:02.260 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376575.log 2024-04-01T05:56:02.261 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210693.log 2024-04-01T05:56:02.261 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388056.log 2024-04-01T05:56:02.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239581.log 2024-04-01T05:56:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564165.log 2024-04-01T05:56:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.389407.log 2024-04-01T05:56:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362746.log 2024-04-01T05:56:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206912.log 2024-04-01T05:56:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440733.log 2024-04-01T05:56:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214527.log 2024-04-01T05:56:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.544588.log 2024-04-01T05:56:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.247394.log 2024-04-01T05:56:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432638.log 2024-04-01T05:56:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.235732.log 2024-04-01T05:56:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201545.log 2024-04-01T05:56:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438188.log 2024-04-01T05:56:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.540551.log 2024-04-01T05:56:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367035.log 2024-04-01T05:56:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204696.log 2024-04-01T05:56:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386662.log 2024-04-01T05:56:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202895.log 2024-04-01T05:56:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420011.log 2024-04-01T05:56:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376566.log 2024-04-01T05:56:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418649.log 2024-04-01T05:56:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471624.log 2024-04-01T05:56:02.276 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452113.log 2024-04-01T05:56:02.277 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.250433.log 2024-04-01T05:56:02.277 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380991.log 2024-04-01T05:56:02.278 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192136.log 2024-04-01T05:56:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212815.log 2024-04-01T05:56:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.263086.log 2024-04-01T05:56:02.280 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216760.log 2024-04-01T05:56:02.280 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234046.log 2024-04-01T05:56:02.280 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127251.log 2024-04-01T05:56:02.281 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228235.log 2024-04-01T05:56:02.282 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128991.log 2024-04-01T05:56:02.282 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.529226.log 2024-04-01T05:56:02.283 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564669.log 2024-04-01T05:56:02.283 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192969.log 2024-04-01T05:56:02.284 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.507029.log 2024-04-01T05:56:02.285 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.412786.log 2024-04-01T05:56:02.285 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.546741.log 2024-04-01T05:56:02.286 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2024-04-01T05:56:02.286 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474305.log 2024-04-01T05:56:02.287 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453385.log 2024-04-01T05:56:02.288 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.547227.log 2024-04-01T05:56:02.288 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443718.log 2024-04-01T05:56:02.289 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473017.log 2024-04-01T05:56:02.290 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132023.log 2024-04-01T05:56:02.290 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.543177.log 2024-04-01T05:56:02.291 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437335.log 2024-04-01T05:56:02.292 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48437.log 2024-04-01T05:56:02.292 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-04-01T05:56:02.293 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.534312.log 2024-04-01T05:56:02.293 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419061.log 2024-04-01T05:56:02.294 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346442.log 2024-04-01T05:56:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.484339.log 2024-04-01T05:56:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.507880.log 2024-04-01T05:56:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491741.log 2024-04-01T05:56:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50432.log 2024-04-01T05:56:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.271682.log 2024-04-01T05:56:02.298 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366622.log 2024-04-01T05:56:02.298 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385706.log 2024-04-01T05:56:02.299 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225894.log 2024-04-01T05:56:02.299 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.563631.log 2024-04-01T05:56:02.300 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128579.log 2024-04-01T05:56:02.300 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.497802.log 2024-04-01T05:56:02.301 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239572.log 2024-04-01T05:56:02.302 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.552028.log 2024-04-01T05:56:02.302 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50768.log 2024-04-01T05:56:02.303 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183957.log 2024-04-01T05:56:02.303 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.272985.log 2024-04-01T05:56:02.304 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.247385.log 2024-04-01T05:56:02.305 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52676.log 2024-04-01T05:56:02.305 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.265713.log 2024-04-01T05:56:02.305 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536969.log 2024-04-01T05:56:02.306 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179652.log 2024-04-01T05:56:02.306 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465929.log 2024-04-01T05:56:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128570.log 2024-04-01T05:56:02.308 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143644.log 2024-04-01T05:56:02.308 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486188.log 2024-04-01T05:56:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.217624.log 2024-04-01T05:56:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.525419.log 2024-04-01T05:56:02.310 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116938.log 2024-04-01T05:56:02.310 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189357.log 2024-04-01T05:56:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202904.log 2024-04-01T05:56:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.543665.log 2024-04-01T05:56:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527952.log 2024-04-01T05:56:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189854.log 2024-04-01T05:56:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187119.log 2024-04-01T05:56:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51774.log 2024-04-01T05:56:02.315 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.563207.log 2024-04-01T05:56:02.315 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433894.log 2024-04-01T05:56:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215436.log 2024-04-01T05:56:02.317 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350677.log 2024-04-01T05:56:02.317 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2024-04-01T05:56:02.318 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144918.log 2024-04-01T05:56:02.319 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377957.log 2024-04-01T05:56:02.319 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134984.log 2024-04-01T05:56:02.320 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.236626.log 2024-04-01T05:56:02.320 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536527.log 2024-04-01T05:56:02.321 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491309.log 2024-04-01T05:56:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243173.log 2024-04-01T05:56:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-04-01T05:56:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465920.log 2024-04-01T05:56:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362335.log 2024-04-01T05:56:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131109.log 2024-04-01T05:56:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.266564.log 2024-04-01T05:56:02.326 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183516.log 2024-04-01T05:56:02.326 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.500406.log 2024-04-01T05:56:02.327 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-04-01T05:56:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-01T05:56:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350686.log 2024-04-01T05:56:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365299.log 2024-04-01T05:56:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436454.log 2024-04-01T05:56:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472481.log 2024-04-01T05:56:02.331 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351103.log 2024-04-01T05:56:02.331 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198854.log 2024-04-01T05:56:02.332 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.496950.log 2024-04-01T05:56:02.333 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.539187.log 2024-04-01T05:56:02.333 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181353.log 2024-04-01T05:56:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468641.log 2024-04-01T05:56:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359814.log 2024-04-01T05:56:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408270.log 2024-04-01T05:56:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216751.log 2024-04-01T05:56:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182206.log 2024-04-01T05:56:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.525428.log 2024-04-01T05:56:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.563198.log 2024-04-01T05:56:02.338 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.387575.log 2024-04-01T05:56:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228226.log 2024-04-01T05:56:02.340 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47831.log 2024-04-01T05:56:02.340 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209438.log 2024-04-01T05:56:02.340 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-04-01T05:56:02.341 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.500397.log 2024-04-01T05:56:02.342 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.538307.log 2024-04-01T05:56:02.342 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260083.log 2024-04-01T05:56:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222455.log 2024-04-01T05:56:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.524295.log 2024-04-01T05:56:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2024-04-01T05:56:02.345 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193861.log 2024-04-01T05:56:02.346 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442020.log 2024-04-01T05:56:02.346 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465000.log 2024-04-01T05:56:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363176.log 2024-04-01T05:56:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527530.log 2024-04-01T05:56:02.348 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.253203.log 2024-04-01T05:56:02.348 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485703.log 2024-04-01T05:56:02.349 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.550688.log 2024-04-01T05:56:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510475.log 2024-04-01T05:56:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.522970.log 2024-04-01T05:56:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209017.log 2024-04-01T05:56:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.360235.log 2024-04-01T05:56:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.492995.log 2024-04-01T05:56:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435174.log 2024-04-01T05:56:02.353 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.563622.log 2024-04-01T05:56:02.353 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.350251.log 2024-04-01T05:56:02.354 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.246930.log 2024-04-01T05:56:02.354 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140071.log 2024-04-01T05:56:02.355 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229580.log 2024-04-01T05:56:02.356 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282115.log 2024-04-01T05:56:02.356 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423109.log 2024-04-01T05:56:02.357 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.499975.log 2024-04-01T05:56:02.357 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-04-01T05:56:02.358 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-04-01T05:56:02.359 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242743.log 2024-04-01T05:56:02.359 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.267397.log 2024-04-01T05:56:02.359 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50558.log 2024-04-01T05:56:02.360 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417942.log 2024-04-01T05:56:02.361 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565919.log 2024-04-01T05:56:02.361 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.560940.log 2024-04-01T05:56:02.361 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506603.log 2024-04-01T05:56:02.362 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125170.log 2024-04-01T05:56:02.363 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211123.log 2024-04-01T05:56:02.363 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136680.log 2024-04-01T05:56:02.364 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.267406.log 2024-04-01T05:56:02.365 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465442.log 2024-04-01T05:56:02.365 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.238305.log 2024-04-01T05:56:02.366 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479141.log 2024-04-01T05:56:02.367 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.532177.log 2024-04-01T05:56:02.367 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.496091.log 2024-04-01T05:56:02.368 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414063.log 2024-04-01T05:56:02.369 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183066.log 2024-04-01T05:56:02.369 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379686.log 2024-04-01T05:56:02.369 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135414.log 2024-04-01T05:56:02.370 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410037.log 2024-04-01T05:56:02.371 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163799.log 2024-04-01T05:56:02.371 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126819.log 2024-04-01T05:56:02.371 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187578.log 2024-04-01T05:56:02.372 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119404.log 2024-04-01T05:56:02.372 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226306.log 2024-04-01T05:56:02.373 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.562266.log 2024-04-01T05:56:02.374 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204246.log 2024-04-01T05:56:02.374 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548499.log 2024-04-01T05:56:02.375 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153046.log 2024-04-01T05:56:02.375 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124686.log 2024-04-01T05:56:02.376 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.524998.log 2024-04-01T05:56:02.377 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234037.log 2024-04-01T05:56:02.377 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375115.log 2024-04-01T05:56:02.378 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119879.log 2024-04-01T05:56:02.379 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49298.log 2024-04-01T05:56:02.379 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195295.log 2024-04-01T05:56:02.380 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.557352.log 2024-04-01T05:56:02.380 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278512.log 2024-04-01T05:56:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565836.log 2024-04-01T05:56:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419070.log 2024-04-01T05:56:02.382 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224193.log 2024-04-01T05:56:02.382 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.391279.log 2024-04-01T05:56:02.383 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564795.log 2024-04-01T05:56:02.384 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361486.log 2024-04-01T05:56:02.384 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.492153.log 2024-04-01T05:56:02.385 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471202.log 2024-04-01T05:56:02.386 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45534.log 2024-04-01T05:56:02.386 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.495249.log 2024-04-01T05:56:02.387 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188450.log 2024-04-01T05:56:02.387 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112281.log 2024-04-01T05:56:02.388 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.551584.log 2024-04-01T05:56:02.388 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.272976.log 2024-04-01T05:56:02.389 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194273.log 2024-04-01T05:56:02.389 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463712.log 2024-04-01T05:56:02.390 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.261788.log 2024-04-01T05:56:02.391 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137532.log 2024-04-01T05:56:02.391 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282124.log 2024-04-01T05:56:02.392 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.263608.log 2024-04-01T05:56:02.392 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474779.log 2024-04-01T05:56:02.393 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.542335.log 2024-04-01T05:56:02.394 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206903.log 2024-04-01T05:56:02.394 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.268240.log 2024-04-01T05:56:02.395 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.424176.log 2024-04-01T05:56:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210271.log 2024-04-01T05:56:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196632.log 2024-04-01T05:56:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128131.log 2024-04-01T05:56:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564992.log 2024-04-01T05:56:02.398 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.389416.log 2024-04-01T05:56:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151773.log 2024-04-01T05:56:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234468.log 2024-04-01T05:56:02.400 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.269933.log 2024-04-01T05:56:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.494307.log 2024-04-01T05:56:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.391784.log 2024-04-01T05:56:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.270805.log 2024-04-01T05:56:02.402 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48293.log 2024-04-01T05:56:02.402 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419599.log 2024-04-01T05:56:02.403 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.412365.log 2024-04-01T05:56:02.404 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.253703.log 2024-04-01T05:56:02.404 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164946.log 2024-04-01T05:56:02.405 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152195.log 2024-04-01T05:56:02.405 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.438179.log 2024-04-01T05:56:02.406 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541478.log 2024-04-01T05:56:02.406 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230599.log 2024-04-01T05:56:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.505116.log 2024-04-01T05:56:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208175.log 2024-04-01T05:56:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137101.log 2024-04-01T05:56:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200203.log 2024-04-01T05:56:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44435.log 2024-04-01T05:56:02.410 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112272.log 2024-04-01T05:56:02.410 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428338.log 2024-04-01T05:56:02.411 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188019.log 2024-04-01T05:56:02.411 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365712.log 2024-04-01T05:56:02.412 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48521.log 2024-04-01T05:56:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46157.log 2024-04-01T05:56:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374662.log 2024-04-01T05:56:02.414 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.494764.log 2024-04-01T05:56:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.478286.log 2024-04-01T05:56:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45540.log 2024-04-01T05:56:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434320.log 2024-04-01T05:56:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.561353.log 2024-04-01T05:56:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140914.log 2024-04-01T05:56:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.207745.log 2024-04-01T05:56:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427477.log 2024-04-01T05:56:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227767.log 2024-04-01T05:56:02.419 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-04-01T05:56:02.419 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.498231.log 2024-04-01T05:56:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381881.log 2024-04-01T05:56:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352372.log 2024-04-01T05:56:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481528.log 2024-04-01T05:56:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-04-01T05:56:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257966.log 2024-04-01T05:56:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506612.log 2024-04-01T05:56:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.349825.log 2024-04-01T05:56:02.424 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.524286.log 2024-04-01T05:56:02.424 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.283386.log 2024-04-01T05:56:02.425 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423118.log 2024-04-01T05:56:02.425 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48742.log 2024-04-01T05:56:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221183.log 2024-04-01T05:56:02.427 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194282.log 2024-04-01T05:56:02.427 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131561.log 2024-04-01T05:56:02.428 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116456.log 2024-04-01T05:56:02.428 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.272113.log 2024-04-01T05:56:02.429 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.558267.log 2024-04-01T05:56:02.430 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2024-04-01T05:56:02.430 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565752.log 2024-04-01T05:56:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225456.log 2024-04-01T05:56:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423530.log 2024-04-01T05:56:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432208.log 2024-04-01T05:56:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.418640.log 2024-04-01T05:56:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49052.log 2024-04-01T05:56:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2024-04-01T05:56:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565583.log 2024-04-01T05:56:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564417.log 2024-04-01T05:56:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.238314.log 2024-04-01T05:56:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115969.log 2024-04-01T05:56:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196623.log 2024-04-01T05:56:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423798.log 2024-04-01T05:56:02.437 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357674.log 2024-04-01T05:56:02.437 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224202.log 2024-04-01T05:56:02.438 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439464.log 2024-04-01T05:56:02.438 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.552907.log 2024-04-01T05:56:02.439 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.256259.log 2024-04-01T05:56:02.440 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222034.log 2024-04-01T05:56:02.440 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230058.log 2024-04-01T05:56:02.441 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260518.log 2024-04-01T05:56:02.441 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.559636.log 2024-04-01T05:56:02.442 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420920.log 2024-04-01T05:56:02.443 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143176.log 2024-04-01T05:56:02.443 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278521.log 2024-04-01T05:56:02.444 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-04-01T05:56:02.444 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130675.log 2024-04-01T05:56:02.445 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197973.log 2024-04-01T05:56:02.446 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120730.log 2024-04-01T05:56:02.447 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.493473.log 2024-04-01T05:56:02.447 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470781.log 2024-04-01T05:56:02.448 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.270814.log 2024-04-01T05:56:02.449 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361495.log 2024-04-01T05:56:02.451 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197082.log 2024-04-01T05:56:02.451 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.261797.log 2024-04-01T05:56:02.452 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490473.log 2024-04-01T05:56:02.452 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407835.log 2024-04-01T05:56:02.452 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151764.log 2024-04-01T05:56:02.453 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352796.log 2024-04-01T05:56:02.454 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472472.log 2024-04-01T05:56:02.455 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.562257.log 2024-04-01T05:56:02.456 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162651.log 2024-04-01T05:56:02.457 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44245.log 2024-04-01T05:56:02.457 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.249090.log 2024-04-01T05:56:02.457 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.242322.log 2024-04-01T05:56:02.458 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239159.log 2024-04-01T05:56:02.458 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233179.log 2024-04-01T05:56:02.459 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.481041.log 2024-04-01T05:56:02.459 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.264459.log 2024-04-01T05:56:02.460 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565331.log 2024-04-01T05:56:02.461 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145351.log 2024-04-01T05:56:02.461 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-01T05:56:02.462 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369995.log 2024-04-01T05:56:02.462 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480554.log 2024-04-01T05:56:02.463 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413618.log 2024-04-01T05:56:02.463 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.360655.log 2024-04-01T05:56:02.463 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.207324.log 2024-04-01T05:56:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.528382.log 2024-04-01T05:56:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346030.log 2024-04-01T05:56:02.465 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344334.log 2024-04-01T05:56:02.465 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358965.log 2024-04-01T05:56:02.466 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348993.log 2024-04-01T05:56:02.467 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216330.log 2024-04-01T05:56:02.467 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385283.log 2024-04-01T05:56:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2024-04-01T05:56:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469508.log 2024-04-01T05:56:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.551575.log 2024-04-01T05:56:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416203.log 2024-04-01T05:56:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2024-04-01T05:56:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452539.log 2024-04-01T05:56:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44287.log 2024-04-01T05:56:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.500828.log 2024-04-01T05:56:02.472 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.264029.log 2024-04-01T05:56:02.472 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506378.log 2024-04-01T05:56:02.473 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46283.log 2024-04-01T05:56:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.389837.log 2024-04-01T05:56:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142744.log 2024-04-01T05:56:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161257.log 2024-04-01T05:56:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139647.log 2024-04-01T05:56:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223781.log 2024-04-01T05:56:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-04-01T05:56:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260936.log 2024-04-01T05:56:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354176.log 2024-04-01T05:56:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50306.log 2024-04-01T05:56:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.493894.log 2024-04-01T05:56:02.479 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146188.log 2024-04-01T05:56:02.479 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218885.log 2024-04-01T05:56:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49582.log 2024-04-01T05:56:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491732.log 2024-04-01T05:56:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.255412.log 2024-04-01T05:56:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.235302.log 2024-04-01T05:56:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.502110.log 2024-04-01T05:56:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415363.log 2024-04-01T05:56:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410028.log 2024-04-01T05:56:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347710.log 2024-04-01T05:56:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436026.log 2024-04-01T05:56:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221613.log 2024-04-01T05:56:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.546319.log 2024-04-01T05:56:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232336.log 2024-04-01T05:56:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195728.log 2024-04-01T05:56:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.508320.log 2024-04-01T05:56:02.487 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.261371.log 2024-04-01T05:56:02.487 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225035.log 2024-04-01T05:56:02.488 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.538298.log 2024-04-01T05:56:02.488 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.421810.log 2024-04-01T05:56:02.488 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229589.log 2024-04-01T05:56:02.489 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189348.log 2024-04-01T05:56:02.489 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.503846.log 2024-04-01T05:56:02.490 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343065.log 2024-04-01T05:56:02.491 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.558730.log 2024-04-01T05:56:02.491 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377051.log 2024-04-01T05:56:02.492 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46890.log 2024-04-01T05:56:02.493 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.273431.log 2024-04-01T05:56:02.493 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201104.log 2024-04-01T05:56:02.494 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185370.log 2024-04-01T05:56:02.494 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229138.log 2024-04-01T05:56:02.496 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.560931.log 2024-04-01T05:56:02.496 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119413.log 2024-04-01T05:56:02.497 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49508.log 2024-04-01T05:56:02.497 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.465433.log 2024-04-01T05:56:02.498 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.492583.log 2024-04-01T05:56:02.499 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181780.log 2024-04-01T05:56:02.499 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436905.log 2024-04-01T05:56:02.500 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476108.log 2024-04-01T05:56:02.501 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564711.log 2024-04-01T05:56:02.501 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228677.log 2024-04-01T05:56:02.502 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.505665.log 2024-04-01T05:56:02.502 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369153.log 2024-04-01T05:56:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452548.log 2024-04-01T05:56:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.449295.log 2024-04-01T05:56:02.504 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482445.log 2024-04-01T05:56:02.504 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157454.log 2024-04-01T05:56:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227194.log 2024-04-01T05:56:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367457.log 2024-04-01T05:56:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.359805.log 2024-04-01T05:56:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180504.log 2024-04-01T05:56:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181362.log 2024-04-01T05:56:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.437757.log 2024-04-01T05:56:02.508 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420486.log 2024-04-01T05:56:02.508 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.249979.log 2024-04-01T05:56:02.508 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366613.log 2024-04-01T05:56:02.509 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377536.log 2024-04-01T05:56:02.509 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.218894.log 2024-04-01T05:56:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367044.log 2024-04-01T05:56:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.533863.log 2024-04-01T05:56:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119888.log 2024-04-01T05:56:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417053.log 2024-04-01T05:56:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219736.log 2024-04-01T05:56:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.543186.log 2024-04-01T05:56:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423846.log 2024-04-01T05:56:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.392288.log 2024-04-01T05:56:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.393762.log 2024-04-01T05:56:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.551141.log 2024-04-01T05:56:02.514 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355470.log 2024-04-01T05:56:02.514 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152186.log 2024-04-01T05:56:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-01T05:56:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510039.log 2024-04-01T05:56:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483493.log 2024-04-01T05:56:02.516 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442862.log 2024-04-01T05:56:02.516 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.498717.log 2024-04-01T05:56:02.516 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452122.log 2024-04-01T05:56:02.517 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45200.log 2024-04-01T05:56:02.517 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130263.log 2024-04-01T05:56:02.518 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153926.log 2024-04-01T05:56:02.518 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448443.log 2024-04-01T05:56:02.518 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510048.log 2024-04-01T05:56:02.519 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.237461.log 2024-04-01T05:56:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.379222.log 2024-04-01T05:56:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375600.log 2024-04-01T05:56:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132866.log 2024-04-01T05:56:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48251.log 2024-04-01T05:56:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202004.log 2024-04-01T05:56:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406496.log 2024-04-01T05:56:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51223.log 2024-04-01T05:56:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145767.log 2024-04-01T05:56:02.524 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.498708.log 2024-04-01T05:56:02.525 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472045.log 2024-04-01T05:56:02.525 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.239150.log 2024-04-01T05:56:02.526 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187569.log 2024-04-01T05:56:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470790.log 2024-04-01T05:56:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45538.log 2024-04-01T05:56:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475642.log 2024-04-01T05:56:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.499553.log 2024-04-01T05:56:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527961.log 2024-04-01T05:56:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541469.log 2024-04-01T05:56:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224614.log 2024-04-01T05:56:02.531 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.376085.log 2024-04-01T05:56:02.531 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45204.log 2024-04-01T05:56:02.532 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.259666.log 2024-04-01T05:56:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430940.log 2024-04-01T05:56:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163808.log 2024-04-01T05:56:02.534 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527117.log 2024-04-01T05:56:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527539.log 2024-04-01T05:56:02.535 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48125.log: /var/log/ceph/ceph-client.admin.74011.log: /var/log/ceph/ceph-client.admin.527117.log: /var/log/ceph/ceph-client.admin.113153.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.356823.log.gz 2024-04-01T05:56:02.536 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.356411.log.gz 2024-04-01T05:56:02.536 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203354.log 2024-04-01T05:56:02.538 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.503397.log 2024-04-01T05:56:02.580 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.193861.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.545025.log.gz 2024-04-01T05:56:02.580 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.272985.log: /var/log/ceph/ceph-client.admin.426192.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.528804.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.505674.log: gzip -5 --verbose 80.8% -- -- replaced with /var/log/ceph/ceph-client.admin.180931.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.411819.log 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.179225.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565877.log: /var/log/ceph/ceph-client.admin.431375.log: /var/log/ceph/ceph-client.admin.423943.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.488107.log: /var/log/ceph/ceph-client.admin.411819.log: /var/log/ceph/ceph-client.admin.482024.log: /var/log/ceph/ceph-client.admin.447180.log 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.212392.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.212806.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.213650.log.gz 2024-04-01T05:56:02.581 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.526263.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.233615.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.481050.log: -- replaced with /var/log/ceph/ceph-client.admin.565877.log.gz 85.8%/var/log/ceph/ceph-client.admin.503397.log: -- replaced with /var/log/ceph/ceph-client.admin.384011.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.232758.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.500819.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.383599.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.542326.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.520293.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.437326.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.542326.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.441154.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.263617.log.gz 2024-04-01T05:56:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.430070.log: /var/log/ceph/ceph-client.admin.507038.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.266143.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.478277.log: gzip -5 --verbose -- 78.7% /var/log/ceph/ceph-client.admin.189845.log 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.237038.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7%/var/log/ceph/ceph-client.admin.427035.log: -- replaced with /var/log/ceph/ceph-client.admin.237883.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.447180.log: /var/log/ceph/ceph-client.admin.525850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129412.log 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.436463.log: 90.2% 90.1% -- replaced with /var/log/ceph/ceph-client.admin.141391.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.496528.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.189845.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.133726.log.gz 2024-04-01T05:56:02.583 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125636.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.434740.log: /var/log/ceph/ceph-client.admin.477371.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.370004.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.391270.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.509597.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.393276.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.504275.log: /var/log/ceph/ceph-client.admin.478707.log: 89.9%/var/log/ceph/ceph-client.admin.464558.log: -- replaced with /var/log/ceph/ceph-client.admin.393285.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.517791.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.484770.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.450592.log 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.495258.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.384862.log.gz 2024-04-01T05:56:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.519009.log: /var/log/ceph/ceph-client.admin.503855.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.136265.log.gz 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565794.log: /var/log/ceph/ceph-client.admin.450592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378378.log 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.485225.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.202454.log.gz 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3%/var/log/ceph/ceph-client.admin.509588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453394.log 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120721.log.gz/var/log/ceph/ceph-client.admin.532186.log: /var/log/ceph/ceph-client.admin.469095.log: /var/log/ceph/ceph-client.admin.463291.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.565794.log.gz 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.378378.log: 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565205.log: 80.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.530922.log 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.536960.log: /var/log/ceph/ceph-client.admin.476538.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.111664.log.gz 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.549790.log: /var/log/ceph/ceph-client.admin.453394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz/var/log/ceph/ceph-client.admin.557786.log: -- replaced with /var/log/ceph/ceph-client.admin.366188.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386200.log 2024-04-01T05:56:02.585 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% 80.4% -- replaced with /var/log/ceph/ceph-client.admin.557786.log.gz 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.536960.log.gz 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1% 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.549790.log.gz 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.530922.log: /var/log/ceph/ceph-client.admin.424167.log: 88.0% 92.5%/var/log/ceph/ceph-client.admin.478698.log: -- replaced with /var/log/ceph/ceph-client.admin.281697.log.gz/var/log/ceph/ceph-client.admin.462027.log: 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.565205.log.gz 82.6% -- replaced with /var/log/ceph/ceph-client.admin.193852.log.gz 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.469086.log: 2024-04-01T05:56:02.586 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.351112.log.gz 2024-04-01T05:56:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.442450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47553.log.gz 2024-04-01T05:56:02.589 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48167.log.gz 2024-04-01T05:56:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.531765.log: /var/log/ceph/ceph-client.admin.434749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343912.log 2024-04-01T05:56:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.488721.log: /var/log/ceph/ceph-client.admin.481537.log: /var/log/ceph/ceph-client.admin.427044.log: /var/log/ceph/ceph-client.admin.564627.log: /var/log/ceph/ceph-client.admin.451886.log: /var/log/ceph/ceph-client.admin.438612.log: 81.3%/var/log/ceph/ceph-client.admin.558258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.564627.log.gz 2024-04-01T05:56:02.589 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.343481.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.463703.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564585.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.115960.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.533442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134142.log 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.343912.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.146661.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.558258.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.557795.log: /var/log/ceph/ceph-client.admin.497793.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.207333.log.gz 2024-04-01T05:56:02.590 INFO:teuthology.orchestra.run.smithi064.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.259235.log.gz 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0%/var/log/ceph/ceph-client.admin.431796.log: -- replaced with /var/log/ceph/ceph-client.admin.259657.log.gz 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.524586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191308.log 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.564585.log.gz 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.480041.log: /var/log/ceph/ceph-client.admin.541899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192978.log 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.546732.log: /var/log/ceph/ceph-client.admin.191308.log: 78.3%/var/log/ceph/ceph-client.admin.532608.log: /var/log/ceph/ceph-client.admin.535186.log: /var/log/ceph/ceph-client.admin.440301.log: /var/log/ceph/ceph-client.admin.549379.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44519.log.gz 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr: 86.0%/var/log/ceph/ceph-client.admin.134142.log: /var/log/ceph/ceph-client.admin.475229.log: /var/log/ceph/ceph-client.admin.430061.log: -- replaced with /var/log/ceph/ceph-client.admin.541899.log.gz/var/log/ceph/ceph-client.admin.508751.log: 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.515046.log 2024-04-01T05:56:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.470360.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.218044.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.469934.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.217633.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.543674.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.549379.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.430488.log: /var/log/ceph/ceph-client.admin.192978.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.546732.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.428785.log: /var/log/ceph/ceph-client.admin.549370.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.198395.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.504266.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.549370.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564249.log: 81.4%/var/log/ceph/ceph-client.admin.539651.log: -- replaced with /var/log/ceph/ceph-client.admin.234881.log.gz 2024-04-01T05:56:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129834.log 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.564249.log.gz 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.531344.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.484761.log: -- /var/log/ceph/ceph-client.admin.464567.log 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45536.log 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.145342.log.gz 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.144056.log.gz/var/log/ceph/ceph-client.admin.510918.log: 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.447189.log: /var/log/ceph/ceph-client.admin.464567.log: 90.2% 84.1% 90.0% -- replaced with /var/log/ceph/ceph-client.admin.421388.log.gz 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.395675.log.gz 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.422269.log.gz/var/log/ceph/ceph-client.admin.564165.log: 2024-04-01T05:56:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.440733.log: /var/log/ceph/ceph-client.admin.544588.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.544588.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.432638.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46367.log 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.564165.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.438188.log: /var/log/ceph/ceph-client.admin.540551.log: /var/log/ceph/ceph-client.admin.45536.log: /var/log/ceph/ceph-client.admin.471624.log: /var/log/ceph/ceph-client.admin.452113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.470369.log 90.2% -- replaced with /var/log/ceph/ceph-client.admin.486197.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% 81.5% -- replaced with /var/log/ceph/ceph-client.admin.540551.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.206487.log.gz 2024-04-01T05:56:02.594 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.477371.log.gz 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%/var/log/ceph/ceph-client.admin.529226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.193861.log.gz 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.433481.log.gz 92.2% -- replaced with /var/log/ceph/ceph-client.admin.557795.log.gz 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564669.log: 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.470369.log: /var/log/ceph/ceph-client.admin.507029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214077.log 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.543674.log.gz 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.546741.log: 90.3%/var/log/ceph/ceph-client.admin.474305.log: -- replaced with /var/log/ceph/ceph-client.admin.406505.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.564669.log.gz 2024-04-01T05:56:02.595 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466904.log 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.453385.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.478277.log.gz 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.547227.log: /var/log/ceph/ceph-client.admin.443718.log: /var/log/ceph/ceph-client.admin.473017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487538.log 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.466904.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.367466.log.gz 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414933.log 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.543177.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.147527.log.gz 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.487538.log: /var/log/ceph/ceph-client.admin.437335.log: /var/log/ceph/ceph-client.admin.534312.log: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.438603.log.gz 2024-04-01T05:56:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.484339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564459.log 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.507880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206062.log 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.491741.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.547227.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564459.log: /var/log/ceph/ceph-client.admin.214077.log: /var/log/ceph/ceph-client.admin.563631.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.497802.log: /var/log/ceph/ceph-client.admin.552028.log: /var/log/ceph/ceph-client.admin.536969.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.564459.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.135836.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.465929.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.536969.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.206062.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.440733.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 90.1% -- replaced with /var/log/ceph/ceph-client.admin.155636.log.gz 2024-04-01T05:56:02.597 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.182637.log 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.486188.log: /var/log/ceph/ceph-client.admin.525419.log: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.214536.log.gz 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.543665.log: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.543665.log.gz 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8%gzip/var/log/ceph/ceph-client.admin.527952.log: -5 --verbose/var/log/ceph/ceph-client.admin.563207.log: -- /var/log/ceph/ceph-client.admin.448869.log 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.563631.log.gz/var/log/ceph/ceph-client.admin.182637.log: 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.433894.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44119.log.gz 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.525007.log 2024-04-01T05:56:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.448869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.272535.log 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.536527.log: /var/log/ceph/ceph-client.admin.525007.log: /var/log/ceph/ceph-client.admin.491309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565373.log 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2%/var/log/ceph/ceph-client.admin.465920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281688.log 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.272535.log: 90.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.526685.log 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.500406.log: /var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.281688.log: gzip 92.9% -- replaced with /var/log/ceph/ceph-client.admin.157000.log.gz 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.200644.log.gz /var/log/ceph/ceph-client.admin.496100.log 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.436454.log: /var/log/ceph/ceph-client.admin.472481.log: /var/log/ceph/ceph-client.admin.496950.log: /var/log/ceph/ceph-client.admin.526685.log: /var/log/ceph/ceph-client.admin.539187.log: /var/log/ceph/ceph-client.admin.468641.log: /var/log/ceph/ceph-client.admin.525428.log: 92.9%/var/log/ceph/ceph-client.admin.563198.log: -- replaced with /var/log/ceph/ceph-client.admin.487105.log.gz 2024-04-01T05:56:02.599 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.447189.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1%/var/log/ceph/ceph-client.admin.500397.log: /var/log/ceph/ceph-client.admin.538307.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.442450.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.441154.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476117.log 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.477801.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.546741.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% 92.3% -- replaced with /var/log/ceph/ceph-client.admin.552028.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.441575.log.gz 2024-04-01T05:56:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.524295.log: 80.4%/var/log/ceph/ceph-client.admin.442020.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.447180.log.gz 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr: -5/var/log/ceph/ceph-client.admin.465000.log: --verbose -- /var/log/ceph/ceph-client.admin.254551.log 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.129000.log.gz/var/log/ceph/ceph-client.admin.476117.log: 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.527530.log: /var/log/ceph/ceph-client.admin.496100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183966.log 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.508742.log.gz 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.485703.log: -- replaced with /var/log/ceph/ceph-client.admin.505107.log.gz 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.473026.log.gz -- /var/log/ceph/ceph-client.admin.218473.log 2024-04-01T05:56:02.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.550688.log: /var/log/ceph/ceph-client.admin.510475.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.536527.log.gz 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.183966.log: /var/log/ceph/ceph-client.admin.522970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45779.log 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% 83.5% -- replaced with /var/log/ceph/ceph-client.admin.550688.log.gz 79.1% 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.195719.log.gz 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.218473.log: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.563198.log.gz 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.492995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118313.log 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.538307.log.gz 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.435174.log: /var/log/ceph/ceph-client.admin.563622.log: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.233170.log.gz 2024-04-01T05:56:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161266.log 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348984.log 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.259666.log: /var/log/ceph/ceph-client.admin.270805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439455.log 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.348984.log: /var/log/ceph/ceph-client.admin.238305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473454.log 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.499975.log: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.257541.log.gz 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.439455.log: /var/log/ceph/ceph-client.admin.565919.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.207754.log.gz 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.560940.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.257552.log.gz 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.506603.log: /var/log/ceph/ceph-client.admin.465442.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.231072.log.gz 2024-04-01T05:56:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.479141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.439029.log 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.526694.log.gz 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0%/var/log/ceph/ceph-client.admin.496091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115504.log 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.244042.log.gz 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.565919.log.gz 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.439029.log: /var/log/ceph/ceph-client.admin.562266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.516403.log 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208166.log 2024-04-01T05:56:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.516403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.542758.log 2024-04-01T05:56:02.666 INFO:teuthology.orchestra.run.smithi100.stderr: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-01T05:56:02.695 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.473454.log: /var/log/ceph/ceph-client.admin.532177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143635.log 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.242331.log.gz 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.273440.log 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.143635.log: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.374232.log.gz 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.548499.log: /var/log/ceph/ceph-client.admin.273440.log: /var/log/ceph/ceph-client.admin.412356.log 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.524998.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.252351.log.gz 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 79.7% -- replaced with /var/log/ceph/ceph-client.admin.192127.log.gz 80.6% -- replaced with /var/log/ceph/ceph-client.admin.498222.log.gz 2024-04-01T05:56:02.696 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.495679.log.gz 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.542758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.533872.log 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.557352.log: /var/log/ceph/ceph-client.admin.565836.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.530491.log.gz 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.565836.log.gz 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.548499.log.gz 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.530500.log.gz 89.8% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.530077.log.gz -- replaced with /var/log/ceph/ceph-client.admin.542758.log.gz 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.697 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.533451.log.gz 81.4% -- replaced with /var/log/ceph/ceph-client.admin.483910.log.gz 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564795.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.564795.log.gz/var/log/ceph/ceph-client.admin.492153.log: 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.471202.log: /var/log/ceph/ceph-client.admin.495249.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.272526.log.gz 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.346451.log.gz 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.551584.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.279886.log.gz 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.463712.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.278096.log.gz 80.7% 2024-04-01T05:56:02.698 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.343903.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.474779.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.189845.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.210280.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.557352.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.542335.log: /var/log/ceph/ceph-client.admin.424176.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.162239.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564992.log: /var/log/ceph/ceph-client.admin.494307.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.381452.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.381890.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.564992.log.gz 2024-04-01T05:56:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.438179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49382.log 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.541478.log: 89.9%/var/log/ceph/ceph-client.admin.505116.log: -- replaced with /var/log/ceph/ceph-client.admin.542335.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44561.log 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.533872.log: 90.2%/var/log/ceph/ceph-client.admin.428338.log: /var/log/ceph/ceph-client.admin.44561.log: /var/log/ceph/ceph-client.admin.494764.log: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.532177.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.478286.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.497802.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.434320.log: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.411819.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 82.8% -- replaced with /var/log/ceph/ceph-client.admin.541478.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.543928.log.gz 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548508.log 2024-04-01T05:56:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.561353.log: -- replaced with /var/log/ceph/ceph-client.admin.209859.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 80.9% -- replaced with /var/log/ceph/ceph-client.admin.561353.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.249503.log 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.531765.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.548508.log: /var/log/ceph/ceph-client.admin.427477.log: 88.2% 80.4% -- replaced with /var/log/ceph/ceph-client.admin.361906.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50684.log.gz -- replaced with /var/log/ceph/ceph-client.admin.52002.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% 90.0% -- replaced with /var/log/ceph/ceph-client.admin.548508.log.gz 2024-04-01T05:56:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.498231.log: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.183057.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.481528.log: /var/log/ceph/ceph-client.admin.506612.log: /var/log/ceph/ceph-client.admin.524286.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44375.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.558267.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.527530.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.527117.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565752.log: /var/log/ceph/ceph-client.admin.432208.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.244051.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.565752.log.gz -- replaced with /var/log/ceph/ceph-client.admin.449755.log.gz 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.249503.log: 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565583.log: /var/log/ceph/ceph-client.admin.564417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462870.log 2024-04-01T05:56:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.439464.log: /var/log/ceph/ceph-client.admin.552907.log: /var/log/ceph/ceph-client.admin.559636.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.564417.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.552907.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.411325.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.565583.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% 92.3% -- replaced with /var/log/ceph/ceph-client.admin.156063.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% 79.2% -- replaced with /var/log/ceph/ceph-client.admin.222025.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.551584.log.gz 80.7% -- replaced with /var/log/ceph/ceph-client.admin.559636.log.gz 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.493473.log: /var/log/ceph/ceph-client.admin.470781.log: 85.0% 92.6% -- replaced with /var/log/ceph/ceph-client.admin.128122.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.164496.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%gzip 90.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.347293.log 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154354.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3%/var/log/ceph/ceph-client.admin.490473.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.466895.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.462870.log: -- replaced with /var/log/ceph/ceph-client.admin.436914.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.472472.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.562257.log: /var/log/ceph/ceph-client.admin.370426.log 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.439875.log.gz 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.439038.log.gz/var/log/ceph/ceph-client.admin.481041.log: 2024-04-01T05:56:02.704 INFO:teuthology.orchestra.run.smithi064.stderr: 82.9% -- replaced with /var/log/ceph/ceph-client.admin.562257.log.gz 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.427912.log.gz 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565331.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.565331.log.gz 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.512513.log 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.480554.log: /var/log/ceph/ceph-client.admin.528382.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.248669.log.gz 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.370426.log: /var/log/ceph/ceph-client.admin.469508.log: /var/log/ceph/ceph-client.admin.551575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141817.log 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9%/var/log/ceph/ceph-client.admin.452539.log: /var/log/ceph/ceph-client.admin.500828.log: -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.249512.log 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.506378.log: /var/log/ceph/ceph-client.admin.512513.log: 2024-04-01T05:56:02.705 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.280386.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.480563.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.551575.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3%/var/log/ceph/ceph-client.admin.493894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257975.log 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.491732.log: -- replaced with /var/log/ceph/ceph-client.admin.558267.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.502110.log: /var/log/ceph/ceph-client.admin.141817.log: /var/log/ceph/ceph-client.admin.436026.log: 78.2%/var/log/ceph/ceph-client.admin.546319.log: /var/log/ceph/ceph-client.admin.249512.log: /var/log/ceph/ceph-client.admin.508320.log: -- replaced with /var/log/ceph/ceph-client.admin.140488.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479555.log 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.538298.log: 92.0% 85.3% -- replaced with /var/log/ceph/ceph-client.admin.538298.log.gz 2024-04-01T05:56:02.706 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.449304.log.gz 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487096.log 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.503846.log: /var/log/ceph/ceph-client.admin.558730.log: /var/log/ceph/ceph-client.admin.257975.log: /var/log/ceph/ceph-client.admin.560931.log: /var/log/ceph/ceph-client.admin.479555.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.546319.log.gz/var/log/ceph/ceph-client.admin.465433.log: 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 80.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.560931.log.gz 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose --/var/log/ceph/ceph-client.admin.492583.log: /var/log/ceph/ceph-client.admin.414942.log 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.436905.log: /var/log/ceph/ceph-client.admin.476108.log: /var/log/ceph/ceph-client.admin.564711.log: /var/log/ceph/ceph-client.admin.487096.log: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.526263.log.gz 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%/var/log/ceph/ceph-client.admin.505665.log: -- replaced with /var/log/ceph/ceph-client.admin.427486.log.gz 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.258397.log 2024-04-01T05:56:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.452548.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.564711.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.414942.log: /var/log/ceph/ceph-client.admin.449295.log: /var/log/ceph/ceph-client.admin.482445.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0%/var/log/ceph/ceph-client.admin.437757.log: -- replaced with /var/log/ceph/ceph-client.admin.525850.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.533863.log: /var/log/ceph/ceph-client.admin.543186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131570.log 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0%/var/log/ceph/ceph-client.admin.551141.log: -- replaced with /var/log/ceph/ceph-client.admin.429214.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408261.log 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.258397.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.486619.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.510039.log: /var/log/ceph/ceph-client.admin.483493.log: /var/log/ceph/ceph-client.admin.442862.log: /var/log/ceph/ceph-client.admin.131570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44561.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.498717.log: /var/log/ceph/ceph-client.admin.452122.log: /var/log/ceph/ceph-client.admin.448443.log: /var/log/ceph/ceph-client.admin.510048.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.489986.log.gz 2024-04-01T05:56:02.708 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.543186.log.gz/var/log/ceph/ceph-client.admin.498708.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.282970.log.gz 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.472045.log: gzip/var/log/ceph/ceph-client.admin.470790.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.364430.log 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.475642.log: /var/log/ceph/ceph-client.admin.499553.log: /var/log/ceph/ceph-client.admin.527961.log: /var/log/ceph/ceph-client.admin.541469.log: /var/log/ceph/ceph-client.admin.430940.log: 94.7% 84.2% -- replaced with /var/log/ceph/ceph-client.admin.541469.log.gz 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.142285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.343056.log/var/log/ceph/ceph-client.admin.199304.log: 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.526685.log.gz 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.132032.log: -- replaced with /var/log/ceph/ceph-client.admin.411828.log.gz/var/log/ceph/ceph-client.admin.45905.log: 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.46031.log: /var/log/ceph/ceph-client.admin.265301.log: /var/log/ceph/ceph-client.admin.132454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240906.log 2024-04-01T05:56:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.152614.log: /var/log/ceph/ceph-client.admin.358550.log: /var/log/ceph/ceph-client.admin.184466.log: /var/log/ceph/ceph-client.admin.281275.log: /var/log/ceph/ceph-client.admin.190273.log: /var/log/ceph/ceph-client.admin.49256.log: /var/log/ceph/ceph-client.admin.417933.log: /var/log/ceph/ceph-client.admin.49666.log: /var/log/ceph/ceph-client.admin.282536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.560077.log 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.392785.log: /var/log/ceph/ceph-client.admin.249081.log: /var/log/ceph/ceph-client.admin.231063.log: /var/log/ceph/ceph-client.admin.262644.log: /var/log/ceph/ceph-client.admin.379231.log: /var/log/ceph/ceph-client.admin.349414.log: /var/log/ceph/ceph-client.admin.240906.log: /var/log/ceph/ceph-client.admin.199745.log: /var/log/ceph/ceph-client.admin.135827.log: /var/log/ceph/ceph-client.admin.52308.log: /var/log/ceph/ceph-client.admin.231906.log: 92.2% 92.9% -- replaced with /var/log/ceph/ceph-client.admin.558730.log.gz/var/log/ceph/ceph-client.admin.127710.log: 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.551141.log.gz 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51434.log: /var/log/ceph/ceph-client.admin.279397.log: /var/log/ceph/ceph-client.admin.48209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.526272.log 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.282961.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.482024.log.gz 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45198.log: /var/log/ceph/ceph-client.admin.560077.log: /var/log/ceph/ceph-client.admin.113663.log: gzip/var/log/ceph/ceph-client.admin.357244.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.236617.log 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.380991.log: /var/log/ceph/ceph-client.admin.210693.log: /var/log/ceph/ceph-client.admin.242752.log: /var/log/ceph/ceph-client.admin.51646.log: /var/log/ceph/ceph-client.admin.381000.log: /var/log/ceph/ceph-client.admin.526272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.212815.log: /var/log/ceph/ceph-client.admin.127251.log: /var/log/ceph/ceph-client.admin.44981.log: /var/log/ceph/ceph-client.admin.216760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.537382.log 2024-04-01T05:56:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.160324.log: /var/log/ceph/ceph-client.admin.271682.log: /var/log/ceph/ceph-client.admin.236617.log: /var/log/ceph/ceph-client.admin.188904.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.560077.log.gz/var/log/ceph/ceph-client.admin.252782.log: 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.128579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.533021.log 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.225894.log: /var/log/ceph/ceph-client.admin.370417.log: /var/log/ceph/ceph-client.admin.239572.log: /var/log/ceph/ceph-client.admin.537382.log: /var/log/ceph/ceph-client.0.52676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.507450.log 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50894.log: /var/log/ceph/ceph-client.admin.48926.log: /var/log/ceph/ceph-client.admin.342643.log: /var/log/ceph/ceph-client.admin.377957.log: /var/log/ceph/ceph-client.admin.128570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468210.log 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.232327.log: /var/log/ceph/ceph-client.admin.216751.log: /var/log/ceph/ceph-client.admin.239581.log: /var/log/ceph/ceph-client.admin.184457.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.537382.log.gz 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.533021.log: /var/log/ceph/ceph-client.admin.143644.log: /var/log/ceph/ceph-client.admin.507450.log: /var/log/ceph/ceph-client.admin.47831.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.209017.log: /var/log/ceph/ceph-client.admin.366622.log: -- /var/log/ceph/ceph-client.admin.118548.log 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.217624.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.270805.log.gz 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.198404.log: /var/log/ceph/ceph-client.admin.468210.log: /var/log/ceph/ceph-client.admin.363588.log: /var/log/ceph/ceph-client.admin.346872.log: /var/log/ceph/ceph-client.admin.356402.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.480041.log.gz 2024-04-01T05:56:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.413206.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.476538.log.gz 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.129421.log: /var/log/ceph/ceph-client.admin.241331.log: /var/log/ceph/ceph-client.admin.147954.log: /var/log/ceph/ceph-client.admin.46073.log: /var/log/ceph/ceph-client.admin.116938.log: /var/log/ceph/ceph-client.admin.271260.log: /var/log/ceph/ceph-client.admin.423934.log: /var/log/ceph/ceph-client.admin.349405.log: /var/log/ceph/ceph-client.admin.111673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375591.log 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50432.log: /var/log/ceph/ceph-client.admin.51518.log: /var/log/ceph/ceph-client.admin.361915.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.258397.log.gz 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.226727.log: /var/log/ceph/ceph-client.admin.202904.log: /var/log/ceph/ceph-client.admin.377966.log: /var/log/ceph/ceph-client.admin.160741.log: /var/log/ceph/ceph-client.admin.118557.log: /var/log/ceph/ceph-client.admin.407409.log: /var/log/ceph/ceph-client.admin.247812.log: /var/log/ceph/ceph-client.admin.527539.log: /var/log/ceph/ceph-client.admin.385706.log: /var/log/ceph/ceph-client.admin.50096.log: /var/log/ceph/ceph-client.admin.138376.log: /var/log/ceph/ceph-client.admin.187119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435609.log 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118548.log: /var/log/ceph/ceph-client.admin.115073.log: /var/log/ceph/ceph-client.admin.251939.log: /var/log/ceph/ceph-client.admin.182646.log: /var/log/ceph/ceph-client.admin.265292.log: /var/log/ceph/ceph-client.admin.390282.log: /var/log/ceph/ceph-client.admin.375591.log: /var/log/ceph/ceph-client.admin.215436.log: /var/log/ceph/ceph-client.admin.350251.log: /var/log/ceph/ceph-client.admin.241766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420020.log 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.214977.log: /var/log/ceph/ceph-client.admin.206053.log: /var/log/ceph/ceph-client.admin.355058.log: /var/log/ceph/ceph-client.admin.186705.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.464124.log.gz 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.271691.log: /var/log/ceph/ceph-client.admin.48563.log: /var/log/ceph/ceph-client.admin.217203.log: /var/log/ceph/ceph-client.admin.46115.log: /var/log/ceph/ceph-client.admin.271269.log: /var/log/ceph/ceph-client.admin.354185.log: /var/log/ceph/ceph-client.admin.44771.log: /var/log/ceph/ceph-client.admin.48377.log: gzip/var/log/ceph/ceph-client.admin.50390.log: /var/log/ceph/ceph-client.admin.344756.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.264450.log 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.408706.log: /var/log/ceph/ceph-client.admin.45205.log: /var/log/ceph/ceph-client.admin.420020.log: /var/log/ceph/ceph-client.admin.435609.log: /var/log/ceph/ceph-client.admin.236626.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.465920.log.gz 2024-04-01T05:56:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.195304.log: /var/log/ceph/ceph-client.admin.190809.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.415354.log: /var/log/ceph/ceph-client.admin.52416.log 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52086.log: /var/log/ceph/ceph-client.admin.349834.log: /var/log/ceph/ceph-client.admin.116447.log: /var/log/ceph/ceph-client.admin.423789.log: /var/log/ceph/ceph-client.admin.215877.log: /var/log/ceph/ceph-client.admin.43857.log: /var/log/ceph/ceph-client.admin.379677.log: /var/log/ceph/ceph-client.admin.179661.log: /var/log/ceph/ceph-client.admin.415782.log: /var/log/ceph/ceph-client.admin.204705.log: /var/log/ceph/ceph-client.admin.231484.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.468632.log.gz 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.376076.log: /var/log/ceph/ceph-client.admin.50726.log: /var/log/ceph/ceph-client.admin.264450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365721.log 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.359394.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.464558.log.gz 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.362335.log: /var/log/ceph/ceph-client.admin.48968.log: /var/log/ceph/ceph-client.admin.153471.log: /var/log/ceph/ceph-client.admin.244499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.415791.log 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.130254.log: /var/log/ceph/ceph-client.admin.137523.log: /var/log/ceph/ceph-client.admin.218053.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.503846.log.gz 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.139638.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.504275.log.gz 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.134993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.241340.log 2024-04-01T05:56:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.266564.log: /var/log/ceph/ceph-client.admin.154784.log: /var/log/ceph/ceph-client.admin.48884.log: /var/log/ceph/ceph-client.admin.389407.log: /var/log/ceph/ceph-client.admin.127260.log: /var/log/ceph/ceph-client.admin.47915.log: /var/log/ceph/ceph-client.admin.219315.log: /var/log/ceph/ceph-client.admin.353667.log: /var/log/ceph/ceph-client.admin.49214.log: /var/log/ceph/ceph-client.admin.365299.log: /var/log/ceph/ceph-client.admin.112712.log: /var/log/ceph/ceph-client.admin.251517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.254977.log 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.206912.log: /var/log/ceph/ceph-client.admin.214527.log: /var/log/ceph/ceph-client.admin.367035.log: /var/log/ceph/ceph-client.admin.247394.log: /var/log/ceph/ceph-client.admin.415791.log: /var/log/ceph/ceph-client.admin.406924.log: /var/log/ceph/ceph-client.admin.180088.log: /var/log/ceph/ceph-client.admin.198854.log: /var/log/ceph/ceph-client.admin.280395.log: /var/log/ceph/ceph-client.admin.217212.log: /var/log/ceph/ceph-client.admin.420011.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.209017.log.gz 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.386662.log: /var/log/ceph/ceph-client.admin.51100.log: /var/log/ceph/ceph-client.admin.49808.log: /var/log/ceph/ceph-client.admin.46469.log: /var/log/ceph/ceph-client.admin.351523.log: /var/log/ceph/ceph-client.admin.408270.log: /var/log/ceph/ceph-client.admin.410901.log: /var/log/ceph/ceph-client.admin.134151.log: /var/log/ceph/ceph-client.admin.418649.log: /var/log/ceph/ceph-client.admin.128991.log: /var/log/ceph/ceph-client.admin.346442.log: /var/log/ceph/ceph-client.admin.48437.log: gzip/var/log/ceph/ceph-client.admin.161710.log: /var/log/ceph/ceph-client.admin.139223.log: /var/log/ceph/ceph-client.admin.182206.log: /var/log/ceph/ceph-client.admin.414498.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.482015.log 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.257107.log: /var/log/ceph/ceph-client.admin.183957.log: /var/log/ceph/ceph-client.admin.247385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.383160.log/var/log/ceph/ceph-client.admin.265713.log: 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.254977.log: /var/log/ceph/ceph-client.admin.179652.log: /var/log/ceph/ceph-client.admin.417520.log: /var/log/ceph/ceph-client.admin.251067.log: /var/log/ceph/ceph-client.admin.364018.log: /var/log/ceph/ceph-client.admin.482015.log: /var/log/ceph/ceph-client.admin.154793.log: /var/log/ceph/ceph-client.admin.228226.log: /var/log/ceph/ceph-client.admin.189357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422691.log 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.189854.log: /var/log/ceph/ceph-client.admin.50054.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.442871.log.gz 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.371285.log: /var/log/ceph/ceph-client.admin.388047.log: /var/log/ceph/ceph-client.admin.238736.log: /var/log/ceph/ceph-client.admin.209438.log: /var/log/ceph/ceph-client.admin.134984.log: gzip/var/log/ceph/ceph-client.admin.144918.log: /var/log/ceph/ceph-client.admin.51774.log: -5/var/log/ceph/ceph-client.admin.350677.log: /var/log/ceph/ceph-client.admin.383160.log: --verbose -- /var/log/ceph/ceph-client.admin.234459.log 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.243173.log: /var/log/ceph/ceph-client.admin.131109.log: /var/log/ceph/ceph-client.admin.146197.log: /var/log/ceph/ceph-client.admin.223772.log: /var/log/ceph/ceph-client.admin.260083.log: /var/log/ceph/ceph-client.admin.350686.log: /var/log/ceph/ceph-client.admin.351103.log: /var/log/ceph/ceph-client.admin.183516.log: /var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.181353.log: /var/log/ceph/ceph-client.admin.211114.log 2024-04-01T05:56:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.422691.log: /var/log/ceph/ceph-client.admin.359814.log: /var/log/ceph/ceph-client.admin.351952.log: /var/log/ceph/ceph-client.admin.346863.log: /var/log/ceph/ceph-client.admin.51602.log: /var/log/ceph/ceph-client.admin.222455.log: /var/log/ceph/ceph-client.admin.387575.log: /var/log/ceph/ceph-client.admin.52170.log: /var/log/ceph/ceph-client.admin.44855.log: /var/log/ceph/ceph-client.admin.363176.log: /var/log/ceph/ceph-client.admin.253203.log: gzip/var/log/ceph/ceph-client.admin.353222.log: /var/log/ceph/ceph-client.admin.234459.log: /var/log/ceph/ceph-client.admin.266976.log: -5/var/log/ceph/ceph-client.admin.218464.log: --verbose -- /var/log/ceph/ceph-client.admin.37772.log 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.348136.log: /var/log/ceph/ceph-client.admin.360235.log: /var/log/ceph/ceph-client.admin.211114.log: /var/log/ceph/ceph-client.admin.345603.log: /var/log/ceph/ceph-client.admin.209429.log: /var/log/ceph/ceph-client.admin.153055.log: /var/log/ceph/ceph-client.admin.46325.log: /var/log/ceph/ceph-client.admin.409149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.353658.log 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.415354.log.gz/var/log/ceph/ceph-client.admin.362746.log: 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.235732.log: /var/log/ceph/ceph-client.admin.376575.log: /var/log/ceph/ceph-client.admin.250433.log: /var/log/ceph/ceph-client.admin.376566.log: /var/log/ceph/ceph-client.admin.201545.log: /var/log/ceph/ceph-client.admin.388056.log: /var/log/ceph/ceph-client.admin.37772.log: /var/log/ceph/ceph-client.admin.204696.log: /var/log/ceph/ceph-client.admin.202895.log: /var/log/ceph/ceph-client.admin.263086.log: /var/log/ceph/ceph-client.admin.46241.log: /var/log/ceph/ceph-client.admin.192136.log: /var/log/ceph/ceph-client.admin.132023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.407418.log 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.228235.log: /var/log/ceph/ceph-client.admin.234046.log: /var/log/ceph/ceph-client.admin.50348.log: /var/log/ceph/ceph-client.admin.192969.log: /var/log/ceph/ceph-client.admin.412786.log: gzip/var/log/ceph/ceph-client.admin.419061.log: -5/var/log/ceph/ceph-client.admin.50768.log: /var/log/ceph/ceph-client.admin.129834.log: --verbose/var/log/ceph/ceph-client.admin.407418.log: --/var/log/ceph/ceph-client.admin.515046.log: /var/log/ceph/ceph-client.admin.145776.log 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.543177.log.gz 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565373.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.563207.log.gz 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.254551.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.565373.log.gz 2024-04-01T05:56:02.715 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.563622.log.gz 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.140071.log: --verbose/var/log/ceph/ceph-client.admin.229580.log: /var/log/ceph/ceph-client.admin.45023.log: -- /var/log/ceph/ceph-client.admin.280852.log 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.242743.log: /var/log/ceph/ceph-client.admin.145776.log: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.271260.log.gz 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.271682.log.gz 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.533442.log.gz 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.271691.log.gz 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.246930.log: /var/log/ceph/ceph-client.admin.423109.log: /var/log/ceph/ceph-client.admin.183066.log: /var/log/ceph/ceph-client.admin.135414.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.163799.log: /var/log/ceph/ceph-client.admin.187578.log: /var/log/ceph/ceph-client.admin.118323.log/var/log/ceph/ceph-client.admin.361486.log: 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.282115.log: /var/log/ceph/ceph-client.admin.50558.log: /var/log/ceph/ceph-client.admin.204246.log: /var/log/ceph/ceph-client.admin.226306.log: /var/log/ceph/ceph-client.admin.128131.log: /var/log/ceph/ceph-client.admin.153046.log: /var/log/ceph/ceph-client.admin.375115.log: /var/log/ceph/ceph-client.admin.44645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.562787.log 2024-04-01T05:56:02.716 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.271269.log.gz 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.125170.log: /var/log/ceph/ceph-client.admin.112281.log: /var/log/ceph/ceph-client.admin.389416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194854.log 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49298.log: /var/log/ceph/ceph-client.admin.224193.log: /var/log/ceph/ceph-client.admin.45534.log: /var/log/ceph/ceph-client.admin.188450.log: /var/log/ceph/ceph-client.admin.267397.log: /var/log/ceph/ceph-client.admin.562787.log: /var/log/ceph/ceph-client.admin.136680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47389.log 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.227767.log: /var/log/ceph/ceph-client.admin.234468.log: /var/log/ceph/ceph-client.admin.50810.log: /var/log/ceph/ceph-client.admin.44287.log: /var/log/ceph/ceph-client.admin.267406.log: /var/log/ceph/ceph-client.admin.261371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.521719.log 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.211123.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.562787.log.gz 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.222034.log: /var/log/ceph/ceph-client.2.47389.log: /var/log/ceph/ceph-client.0.46890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.201995.log 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.494307.log.gz 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.229589.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.493894.log.gz 2024-04-01T05:56:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.230058.log: /var/log/ceph/ceph-client.admin.188019.log: /var/log/ceph/ceph-client.admin.45200.log: /var/log/ceph/ceph-client.admin.119413.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.494764.log.gz 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.521719.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.484348.log.gz 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.260518.log: /var/log/ceph/ceph-client.admin.227194.log: /var/log/ceph/ceph-client.admin.153926.log: /var/log/ceph/ceph-client.admin.270814.log: /var/log/ceph/ceph-client.admin.48521.log: /var/log/ceph/ceph-client.admin.386200.log: /var/log/ceph/ceph-client.admin.414063.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.367457.log: -- /var/log/ceph/ceph-client.admin.426609.log 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.278521.log: /var/log/ceph/ceph-client.admin.201995.log: /var/log/ceph/ceph-client.admin.359805.log: /var/log/ceph/ceph-client.admin.180504.log: /var/log/ceph/ceph-client.admin.145351.log: /var/log/ceph/ceph-client.admin.374662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.492162.log 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.181362.log: 81.5%/var/log/ceph/ceph-client.admin.420486.log: /var/log/ceph/ceph-client.admin.379686.log: /var/log/ceph/ceph-client.admin.249979.log: /var/log/ceph/ceph-client.admin.377536.log: /var/log/ceph/ceph-client.admin.361495.log: -- replaced with /var/log/ceph/ceph-client.admin.487096.log.gz 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.272113.log: /var/log/ceph/ceph-client.admin.346030.log: /var/log/ceph/ceph-client.admin.218894.log: /var/log/ceph/ceph-client.admin.367044.log: /var/log/ceph/ceph-client.admin.410037.log: /var/log/ceph/ceph-client.admin.417053.log: /var/log/ceph/ceph-client.admin.426609.log: /var/log/ceph/ceph-client.admin.119888.log: /var/log/ceph/ceph-client.admin.261797.log: /var/log/ceph/ceph-client.admin.194273.log: gzip/var/log/ceph/ceph-client.admin.126819.log: -5/var/log/ceph/ceph-client.admin.423846.log: --verbose -- /var/log/ceph/ceph-client.admin.266985.log 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.219736.log: /var/log/ceph/ceph-client.admin.143176.log: /var/log/ceph/ceph-client.admin.358965.log: /var/log/ceph/ceph-client.admin.392288.log: /var/log/ceph/ceph-client.admin.492162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.254560.log 2024-04-01T05:56:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.393762.log: /var/log/ceph/ceph-client.admin.261788.log: /var/log/ceph/ceph-client.admin.185370.log: /var/log/ceph/ceph-client.admin.119404.log: /var/log/ceph/ceph-client.admin.44939.log: /var/log/ceph/ceph-client.admin.355470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.560068.log 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.130675.log: /var/log/ceph/ceph-client.admin.266985.log: /var/log/ceph/ceph-client.admin.264029.log: /var/log/ceph/ceph-client.admin.132866.log: /var/log/ceph/ceph-client.admin.379222.log: /var/log/ceph/ceph-client.admin.124686.log: /var/log/ceph/ceph-client.admin.282124.log: /var/log/ceph/ceph-client.admin.130263.log: /var/log/ceph/ceph-client.admin.376085.log: /var/log/ceph/ceph-client.admin.202004.log: /var/log/ceph/ceph-client.admin.260936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429640.log 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.120730.log: /var/log/ceph/ceph-client.admin.45194.log: /var/log/ceph/ceph-client.admin.46367.log: /var/log/ceph/ceph-client.admin.234037.log: /var/log/ceph/ceph-client.admin.206903.log: /var/log/ceph/ceph-client.admin.560068.log: /var/log/ceph/ceph-client.admin.237461.log: /var/log/ceph/ceph-client.admin.210271.log: /var/log/ceph/ceph-client.admin.254560.log: /var/log/ceph/ceph-client.admin.151764.log: /var/log/ceph/ceph-client.admin.50306.log: gzip/var/log/ceph/ceph-client.admin.151773.log: /var/log/ceph/ceph-client.admin.414933.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.270363.log 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.119879.log: /var/log/ceph/ceph-client.admin.375600.log: /var/log/ceph/ceph-client.admin.391784.log: /var/log/ceph/ceph-client.admin.269933.log: /var/log/ceph/ceph-client.admin.429640.log: /var/log/ceph/ceph-client.admin.196632.log: 80.6%/var/log/ceph/ceph-client.admin.389837.log: /var/log/ceph/ceph-client.admin.235302.log: -- replaced with /var/log/ceph/ceph-client.admin.560068.log.gz 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.345182.log.gz 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48293.log: /var/log/ceph/ceph-client.admin.195295.log: /var/log/ceph/ceph-client.admin.253703.log: /var/log/ceph/ceph-client.admin.48251.log: /var/log/ceph/ceph-client.admin.201104.log: gzip/var/log/ceph/ceph-client.admin.139647.log: -5 --verbose/var/log/ceph/ceph-client.admin.278512.log: /var/log/ceph/ceph-client.admin.412365.log: /var/log/ceph/ceph-client.admin.137101.log: -- /var/log/ceph/ceph-client.admin.368301.log/var/log/ceph/ceph-client.admin.229138.log: 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.406496.log: /var/log/ceph/ceph-client.admin.152195.log: /var/log/ceph/ceph-client.admin.200203.log: /var/log/ceph/ceph-client.admin.44435.log: /var/log/ceph/ceph-client.admin.366613.log: /var/log/ceph/ceph-client.admin.419070.log: /var/log/ceph/ceph-client.admin.365712.log: /var/log/ceph/ceph-client.admin.46157.log: /var/log/ceph/ceph-client.admin.49508.log: /var/log/ceph/ceph-client.admin.45540.log: /var/log/ceph/ceph-client.admin.140914.log: /var/log/ceph/ceph-client.admin.145767.log: /var/log/ceph/ceph-client.admin.391279.log: /var/log/ceph/ceph-client.admin.51223.log: /var/log/ceph/ceph-client.admin.49172.log: /var/log/ceph/ceph-client.admin.381881.log: /var/log/ceph/ceph-client.admin.207745.log: /var/log/ceph/ceph-client.admin.152186.log: /var/log/ceph/ceph-client.admin.257966.log: /var/log/ceph/ceph-client.admin.239150.log: /var/log/ceph/ceph-client.admin.349825.log: /var/log/ceph/ceph-client.admin.187569.log: /var/log/ceph/ceph-client.admin.272976.log: /var/log/ceph/ceph-client.admin.417942.log: /var/log/ceph/ceph-client.admin.112272.log: /var/log/ceph/ceph-client.admin.194282.log: /var/log/ceph/ceph-client.admin.48742.log: /var/log/ceph/ceph-client.admin.283386.log: gzip/var/log/ceph/ceph-client.admin.224614.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.519000.log 2024-04-01T05:56:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.131561.log: /var/log/ceph/ceph-client.admin.116456.log: /var/log/ceph/ceph-client.admin.137532.log: /var/log/ceph/ceph-client.admin.268240.log: /var/log/ceph/ceph-client.admin.423530.log: /var/log/ceph/ceph-client.admin.225456.log: /var/log/ceph/ceph-client.admin.48658.log: /var/log/ceph/ceph-client.admin.45204.log: /var/log/ceph/ceph-client.admin.418640.log: /var/log/ceph/ceph-client.admin.49052.log: /var/log/ceph/ceph-client.admin.164946.log: /var/log/ceph/ceph-client.admin.263608.log: /var/log/ceph/ceph-client.admin.115969.log: /var/log/ceph/ceph-client.admin.44729.log: /var/log/ceph/ceph-client.admin.238314.log: /var/log/ceph/ceph-client.admin.357674.log: /var/log/ceph/ceph-client.admin.423798.log: /var/log/ceph/ceph-client.admin.230599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453806.log 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.419599.log: /var/log/ceph/ceph-client.admin.196623.log: /var/log/ceph/ceph-client.admin.224202.log: /var/log/ceph/ceph-client.admin.256259.log: /var/log/ceph/ceph-client.admin.48826.log: /var/log/ceph/ceph-client.admin.197973.log: /var/log/ceph/ceph-client.admin.352796.log: /var/log/ceph/ceph-client.admin.368301.log: /var/log/ceph/ceph-client.admin.208175.log: /var/log/ceph/ceph-client.admin.364430.log: /var/log/ceph/ceph-client.admin.408261.log: /var/log/ceph/ceph-client.admin.420920.log: /var/log/ceph/ceph-client.admin.197082.log: /var/log/ceph/ceph-client.admin.407835.log: gzip/var/log/ceph/ceph-client.admin.249090.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.445616.log 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44245.log: /var/log/ceph/ceph-client.admin.46283.log: /var/log/ceph/ceph-client.admin.352372.log: /var/log/ceph/ceph-client.admin.162651.log: /var/log/ceph/ceph-client.admin.239159.log: /var/log/ceph/ceph-client.admin.242322.log: /var/log/ceph/ceph-client.admin.218885.log: /var/log/ceph/ceph-client.admin.519000.log: /var/log/ceph/ceph-client.admin.48335.log: /var/log/ceph/ceph-client.admin.233179.log: 80.5%/var/log/ceph/ceph-client.admin.369995.log: /var/log/ceph/ceph-client.admin.423118.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.506612.log.gz 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.264459.log: -- replaced with /var/log/ceph/ceph-client.admin.507029.log.gz/var/log/ceph/ceph-client.admin.415363.log: /var/log/ceph/ceph-client.admin.413618.log: 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.207324.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.507450.log.gz 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.360655.log: /var/log/ceph/ceph-client.admin.221183.log: /var/log/ceph/ceph-client.admin.347710.log: /var/log/ceph/ceph-client.admin.344334.log: /var/log/ceph/ceph-client.admin.385283.log: /var/log/ceph/ceph-client.admin.348993.log: /var/log/ceph/ceph-client.admin.50516.log: /var/log/ceph/ceph-client.admin.216330.log: /var/log/ceph/ceph-client.admin.221613.log: /var/log/ceph/ceph-client.admin.142744.log: /var/log/ceph/ceph-client.admin.416203.log: /var/log/ceph/ceph-client.admin.161257.log: /var/log/ceph/ceph-client.admin.232336.log: 90.2%/var/log/ceph/ceph-client.admin.46199.log: -- replaced with /var/log/ceph/ceph-client.admin.507038.log.gz 2024-04-01T05:56:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.146188.log: /var/log/ceph/ceph-client.admin.354176.log: /var/log/ceph/ceph-client.admin.453806.log: /var/log/ceph/ceph-client.admin.203354.log: /var/log/ceph/ceph-client.admin.223781.log: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.430061.log.gz 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49582.log: /var/log/ceph/ceph-client.admin.410028.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.430940.log.gz 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.430488.log.gz 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.255412.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.430070.log.gz 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.129412.log: /var/log/ceph/ceph-client.admin.195728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.547648.log 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.421810.log: /var/log/ceph/ceph-client.admin.225035.log: /var/log/ceph/ceph-client.admin.343065.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.432638.log.gz 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.445616.log: /var/log/ceph/ceph-client.admin.189348.log: /var/log/ceph/ceph-client.admin.45527.log: /var/log/ceph/ceph-client.admin.377051.log: /var/log/ceph/ceph-client.admin.273431.log: gzip -5/var/log/ceph/ceph-client.admin.181780.log: --verbose/var/log/ceph/ceph-client.admin.547648.log: -- /var/log/ceph/ceph-client.admin.131100.log/var/log/ceph/ceph-client.admin.228677.log: /var/log/ceph/ceph-client.admin.157454.log: 2024-04-01T05:56:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.369153.log: /var/log/ceph/ceph-client.admin.45538.log: /var/log/ceph/ceph-client.admin.163808.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.547648.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.431375.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.560940.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.343056.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.465929.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45537.log 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.503855.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.206053.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.504266.log.gz 2024-04-01T05:56:02.723 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52416.log: /var/log/ceph/ceph-client.admin.353658.log: /var/log/ceph/ceph-client.admin.118323.log: /var/log/ceph/ceph-client.admin.270363.log: /var/log/ceph/ceph-client.admin.45947.log: /var/log/ceph/ceph-client.admin.45779.log: gzip/var/log/ceph/ceph-client.admin.161266.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.489133.log 2024-04-01T05:56:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.208166.log: /var/log/ceph/ceph-client.admin.118313.log: /var/log/ceph/ceph-client.admin.131100.log: /var/log/ceph/ceph-client.admin.412356.log: /var/log/ceph/ceph-client.admin.50978.log: /var/log/ceph/ceph-client.admin.365721.log: /var/log/ceph/ceph-client.admin.241340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.432217.log 2024-04-01T05:56:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.347293.log: /var/log/ceph/ceph-client.admin.280852.log: /var/log/ceph/ceph-client.admin.194854.log: /var/log/ceph/ceph-client.admin.45537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.450601.log 2024-04-01T05:56:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489133.log: /var/log/ceph/ceph-client.admin.432217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.441145.log 2024-04-01T05:56:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.450601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352363.log 2024-04-01T05:56:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.441145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416634.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.352363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125161.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.416634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364873.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.125161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50852.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.364873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51020.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211960.log 2024-04-01T05:56:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183507.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.211960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205156.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.183507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425766.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.205156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.366197.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.425766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.266134.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.366197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429223.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.266134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164075.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.429223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.442441.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.229129.log 2024-04-01T05:56:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.164075.log: /var/log/ceph/ceph-client.admin.442441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.347719.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.251058.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.229129.log: /var/log/ceph/ceph-client.admin.347719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222867.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151285.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.251058.log: /var/log/ceph/ceph-client.admin.222867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.475651.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357253.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.151285.log: /var/log/ceph/ceph-client.admin.475651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144909.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147075.log 2024-04-01T05:56:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.357253.log: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.500397.log.gz 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.144909.log: gzip 84.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.500819.log.gz -- 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.534754.log 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.147075.log: /var/log/ceph/ceph-client.admin.363167.log 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.534754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.250424.log 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%/var/log/ceph/ceph-client.admin.363167.log: -- replaced with /var/log/ceph/ceph-client.admin.500406.log.gz 90.3% -- replaced with /var/log/ceph/ceph-client.admin.500828.log.gz 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.466407.log 2024-04-01T05:56:02.749 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.426609.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.250424.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.427477.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.427035.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.485225.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 90.2% -- replaced with /var/log/ceph/ceph-client.admin.427044.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose 90.1% -- replaced with /var/log/ceph/ceph-client.admin.426192.log.gz 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.488098.log 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.466407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.206478.log 2024-04-01T05:56:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.488098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118983.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.206478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462861.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.540106.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.462861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369583.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.540106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.556021.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.369583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565037.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.556021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50180.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197532.log/var/log/ceph/ceph-client.admin.50180.log: 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.371276.log 2024-04-01T05:56:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.565037.log.gz 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.197532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420495.log 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.420495.log: /var/log/ceph/ceph-client.admin.197073.log 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.197073.log: /var/log/ceph/ceph-client.admin.49624.log 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3%/var/log/ceph/ceph-client.admin.371276.log: -- replaced with /var/log/ceph/ceph-client.admin.434320.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.494773.log 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.752 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.434740.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.435174.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49624.log: /var/log/ceph/ceph-client.admin.494773.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.437326.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.438179.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564543.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.434749.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.437335.log.gz 2024-04-01T05:56:02.753 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 92.2% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.556021.log.gz -- replaced with /var/log/ceph/ceph-client.admin.50852.log.gz 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.113153.log.gz 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.529638.log 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564543.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.564543.log.gz 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.565626.log 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.529638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.249988.log 2024-04-01T05:56:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.363597.log 2024-04-01T05:56:02.755 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.192978.log.gz 2024-04-01T05:56:02.755 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.565626.log.gz 2024-04-01T05:56:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.249988.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.481050.log.gz 2024-04-01T05:56:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48083.log 2024-04-01T05:56:02.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.363597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137110.log 2024-04-01T05:56:02.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510466.log 2024-04-01T05:56:02.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.137110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.547218.log 81.1% 2024-04-01T05:56:02.764 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.128570.log.gz 2024-04-01T05:56:02.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.510466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139214.log 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.547218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114154.log 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.547218.log.gz 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565710.log 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114154.log: /var/log/ceph/ceph-client.admin.139214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115064.log 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410910.log/var/log/ceph/ceph-client.admin.115064.log: 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.283395.log 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.565710.log.gz 2024-04-01T05:56:02.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.410910.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.272985.log.gz 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364009.log 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.283395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.264038.log 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.364009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.431366.log 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.264038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.486610.log 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.431366.log: gzip 81.2% -- replaced with /var/log/ceph/ceph-client.admin.145767.log.gz 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.486610.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.478698.log.gz 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose 78.0% -- replaced with /var/log/ceph/ceph-client.admin.146188.log.gz 2024-04-01T05:56:02.766 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.153935.log 2024-04-01T05:56:02.767 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.264459.log.gz 2024-04-01T05:56:02.767 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% 78.9% -- replaced with /var/log/ceph/ceph-client.admin.264450.log.gz 2024-04-01T05:56:02.767 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.145776.log.gz 2024-04-01T05:56:02.774 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.264038.log.gz 2024-04-01T05:56:02.774 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.556457.log 2024-04-01T05:56:02.774 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.474305.log.gz 2024-04-01T05:56:02.774 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.153935.log: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.474779.log.gz 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417511.log 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.469095.log.gz 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.556457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354628.log 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.556457.log.gz 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.417511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.370860.log 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.250424.log.gz 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.354628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185804.log 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.152186.log.gz 2024-04-01T05:56:02.775 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.151773.log.gz 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.370860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384431.log 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr: 78.9% -- replaced with /var/log/ceph/ceph-client.admin.466407.log.gz 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.185804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.423109.log.gz 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.384431.log: --verbose -- /var/log/ceph/ceph-client.admin.416212.log 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.204255.log 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.423118.log.gz 2024-04-01T05:56:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.416212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.254986.log 2024-04-01T05:56:02.777 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.416625.log 2024-04-01T05:56:02.782 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.204255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.520302.log 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.254986.log: /var/log/ceph/ceph-client.admin.416625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134563.log 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.520302.log: /var/log/ceph/ceph-client.admin.47957.log 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.214077.log.gz 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.386671.log 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.134563.log: /var/log/ceph/ceph-client.admin.47957.log: 78.9% -- replaced with /var/log/ceph/ceph-client.admin.139638.log.gz 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.139223.log.gz 2024-04-01T05:56:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.497372.log 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.486610.log.gz/var/log/ceph/ceph-client.admin.386671.log: 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.200653.log 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.139647.log.gz 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.497372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278950.log 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.200653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423894.log 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.278950.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.413197.log 2024-04-01T05:56:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.423894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.463282.log 2024-04-01T05:56:02.787 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443709.log 2024-04-01T05:56:02.787 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.413197.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.463282.log: /var/log/ceph/ceph-client.admin.564753.log 2024-04-01T05:56:02.787 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.443709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186246.log 2024-04-01T05:56:02.787 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.480050.log 2024-04-01T05:56:02.787 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.186246.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.262635.log 2024-04-01T05:56:02.788 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.564753.log.gz 2024-04-01T05:56:02.788 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.480050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118992.log 2024-04-01T05:56:02.789 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.262635.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.138367.log 2024-04-01T05:56:02.789 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.471624.log.gz 2024-04-01T05:56:02.789 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.265722.log 2024-04-01T05:56:02.790 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.138367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490464.log 2024-04-01T05:56:02.790 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.265722.log: /var/log/ceph/ceph-client.admin.118992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.540560.log 2024-04-01T05:56:02.791 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.254134.log/var/log/ceph/ceph-client.admin.490464.log: /var/log/ceph/ceph-client.admin.540560.log: 2024-04-01T05:56:02.792 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-04-01T05:56:02.792 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.412777.log 2024-04-01T05:56:02.793 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.254134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422260.log 2024-04-01T05:56:02.795 INFO:teuthology.orchestra.run.smithi100.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-01T05:56:02.796 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.422260.log: /var/log/ceph/ceph-client.admin.412777.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.497793.log.gz 2024-04-01T05:56:02.797 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.233624.log 2024-04-01T05:56:02.797 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.493473.log.gz 2024-04-01T05:56:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448878.log 2024-04-01T05:56:02.799 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.233624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.387584.log 2024-04-01T05:56:02.800 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.448878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474314.log 2024-04-01T05:56:02.801 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.387584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506114.log 2024-04-01T05:56:02.803 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.474314.log: -- /var/log/ceph/ceph-client.admin.499141.log 2024-04-01T05:56:02.804 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.506114.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.138791.log 2024-04-01T05:56:02.805 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.499141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141382.log 2024-04-01T05:56:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.138791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373820.log 2024-04-01T05:56:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.264880.log 2024-04-01T05:56:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.141382.log: /var/log/ceph/ceph-client.admin.373820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.420911.log 2024-04-01T05:56:02.808 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126828.log 2024-04-01T05:56:02.809 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.420911.log: /var/log/ceph/ceph-client.admin.264880.log: gzip 80.8% -- replaced with /var/log/ceph/ceph-client.admin.527952.log.gz 2024-04-01T05:56:02.809 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.560509.log 2024-04-01T05:56:02.810 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.126828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394238.log 2024-04-01T05:56:02.810 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.529226.log.gz 2024-04-01T05:56:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.560509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132875.log 2024-04-01T05:56:02.812 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.560509.log.gz/var/log/ceph/ceph-client.admin.394238.log: 2024-04-01T05:56:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129843.log 2024-04-01T05:56:02.813 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.132875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.256681.log 2024-04-01T05:56:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191317.log/var/log/ceph/ceph-client.admin.129843.log: /var/log/ceph/ceph-client.admin.256681.log: 2024-04-01T05:56:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.191317.log: /var/log/ceph/ceph-client.admin.550697.log 2024-04-01T05:56:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.545459.log/var/log/ceph/ceph-client.admin.550697.log: 2024-04-01T05:56:02.817 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.413627.log 2024-04-01T05:56:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.545459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141808.log 2024-04-01T05:56:02.821 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.413627.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.545459.log.gz 2024-04-01T05:56:02.821 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140497.log 2024-04-01T05:56:02.821 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% 81.4% -- replaced with /var/log/ceph/ceph-client.admin.354628.log.gz 2024-04-01T05:56:02.822 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.550697.log.gz 2024-04-01T05:56:02.822 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.141808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225465.log 2024-04-01T05:56:02.823 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51732.log 2024-04-01T05:56:02.824 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.140497.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.51732.log: -- /var/log/ceph/ceph-client.admin.51350.log 2024-04-01T05:56:02.825 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.225465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133717.log 2024-04-01T05:56:02.826 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211546.log 2024-04-01T05:56:02.827 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 81.1% -- replaced with /var/log/ceph/ceph-client.admin.448022.log.gz 2024-04-01T05:56:02.827 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose/var/log/ceph/ceph-client.admin.211546.log: -- /var/log/ceph/ceph-client.admin.223349.log 2024-04-01T05:56:02.828 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.449746.log.gz 2024-04-01T05:56:02.828 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.450172.log.gz 2024-04-01T05:56:02.828 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.450592.log.gz 2024-04-01T05:56:02.828 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.452113.log.gz 2024-04-01T05:56:02.829 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.448031.log.gz 2024-04-01T05:56:02.829 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.133717.log: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.506114.log.gz 2024-04-01T05:56:02.830 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.450181.log.gz 2024-04-01T05:56:02.831 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 80.7% -- replaced with /var/log/ceph/ceph-client.admin.440301.log.gz 2024-04-01T05:56:02.831 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.190800.log 2024-04-01T05:56:02.831 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.223349.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.452548.log.gz 2024-04-01T05:56:02.832 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.549799.log 2024-04-01T05:56:02.833 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.190800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.440724.log 2024-04-01T05:56:02.834 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.549799.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.470360.log.gz 2024-04-01T05:56:02.834 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.470369.log.gz 2024-04-01T05:56:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.440724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536536.log 2024-04-01T05:56:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565163.log 2024-04-01T05:56:02.837 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.536536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162230.log 2024-04-01T05:56:02.838 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.384431.log.gz 2024-04-01T05:56:02.839 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.370860.log.gz 2024-04-01T05:56:02.840 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565163.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.153935.log.gz 2024-04-01T05:56:02.841 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.565163.log.gz 2024-04-01T05:56:02.841 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.433894.log.gz 2024-04-01T05:56:02.843 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489142.log 2024-04-01T05:56:02.843 INFO:teuthology.orchestra.run.smithi064.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.516403.log.gz 2024-04-01T05:56:02.844 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.162230.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.536536.log.gz 2024-04-01T05:56:02.845 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2024-04-01T05:56:02.846 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489142.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.423943.log.gz 2024-04-01T05:56:02.846 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.364009.log.gz 2024-04-01T05:56:02.846 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-04-01T05:56:02.847 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.419590.log 2024-04-01T05:56:02.847 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.363597.log.gz 2024-04-01T05:56:02.849 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213659.log 2024-04-01T05:56:02.850 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.419590.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.510466.log.gz 2024-04-01T05:56:02.852 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 81.4% -- replaced with /var/log/ceph/ceph-client.admin.142276.log.gz 2024-04-01T05:56:02.852 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.153480.log 81.7% -- replaced with /var/log/ceph/ceph-client.admin.141808.log.gz 2024-04-01T05:56:02.852 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.853 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.213659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.346021.log 2024-04-01T05:56:02.854 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.153480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.540097.log 2024-04-01T05:56:02.856 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.422260.log.gz 2024-04-01T05:56:02.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.540097.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.540097.log.gz 2024-04-01T05:56:02.858 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565457.log 2024-04-01T05:56:02.859 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.346021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.219306.log 2024-04-01T05:56:02.859 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.218473.log.gz 2024-04-01T05:56:02.860 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.408261.log.gz 2024-04-01T05:56:02.860 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.408706.log.gz 2024-04-01T05:56:02.864 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.410028.log.gz 2024-04-01T05:56:02.864 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% 90.3% -- replaced with /var/log/ceph/ceph-client.admin.429223.log.gz 2024-04-01T05:56:02.864 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.481537.log.gz 2024-04-01T05:56:02.865 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565457.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.565457.log.gz 2024-04-01T05:56:02.867 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 90.1% -- replaced with /var/log/ceph/ceph-client.admin.239159.log.gz 2024-04-01T05:56:02.867 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.221192.log 2024-04-01T05:56:02.868 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133301.log 2024-04-01T05:56:02.868 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.281688.log.gz 2024-04-01T05:56:02.870 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.221192.log: /var/log/ceph/ceph-client.admin.133301.log: /var/log/ceph/ceph-client.admin.219306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.254125.log 2024-04-01T05:56:02.870 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.495670.log 2024-04-01T05:56:02.872 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.254125.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.527961.log.gz 2024-04-01T05:56:02.872 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.501247.log 2024-04-01T05:56:02.872 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.495670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.430497.log 2024-04-01T05:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.209850.log 2024-04-01T05:56:02.876 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.501247.log: /var/log/ceph/ceph-client.admin.430497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.258388.log 2024-04-01T05:56:02.876 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49424.log 2024-04-01T05:56:02.876 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.209850.log: /var/log/ceph/ceph-client.admin.258388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154364.log 2024-04-01T05:56:02.876 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152623.log 2024-04-01T05:56:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49424.log: /var/log/ceph/ceph-client.admin.154364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185361.log 2024-04-01T05:56:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260945.log 2024-04-01T05:56:02.879 INFO:teuthology.orchestra.run.smithi064.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.197073.log.gz 2024-04-01T05:56:02.879 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4%/var/log/ceph/ceph-client.admin.185361.log: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.539187.log.gz 2024-04-01T05:56:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.152623.log: -- replaced with /var/log/ceph/ceph-client.admin.443709.log.gz 2024-04-01T05:56:02.880 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.447602.log 2024-04-01T05:56:02.880 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.260945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.544597.log 2024-04-01T05:56:02.880 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.447602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.562778.log 2024-04-01T05:56:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.544597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.510927.log 2024-04-01T05:56:02.883 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.544597.log.gz 2024-04-01T05:56:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.562778.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.255829.log 2024-04-01T05:56:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.510927.log: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.562778.log.gz 2024-04-01T05:56:02.884 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.267827.log 2024-04-01T05:56:02.886 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.255829.log: gzip -5 --verbose 80.0% -- replaced with /var/log/ceph/ceph-client.admin.510918.log.gz 2024-04-01T05:56:02.886 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.256250.log 2024-04-01T05:56:02.886 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.267827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.559192.log 2024-04-01T05:56:02.886 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.559192.log: /var/log/ceph/ceph-client.admin.253712.log 2024-04-01T05:56:02.887 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.256250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489555.log 2024-04-01T05:56:02.887 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.213659.log.gz/var/log/ceph/ceph-client.admin.253712.log: 2024-04-01T05:56:02.889 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348145.log 2024-04-01T05:56:02.890 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121152.log 2024-04-01T05:56:02.892 INFO:teuthology.orchestra.run.smithi064.stderr: 79.8% 80.9% -- replaced with /var/log/ceph/ceph-client.admin.346021.log.gz 2024-04-01T05:56:02.892 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.162230.log.gz 2024-04-01T05:56:02.893 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.348145.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.369583.log.gz 2024-04-01T05:56:02.893 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.559192.log.gz 2024-04-01T05:56:02.894 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461558.log 2024-04-01T05:56:02.895 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358095.log 2024-04-01T05:56:02.895 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.121152.log: /var/log/ceph/ceph-client.admin.461558.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.510475.log.gz 2024-04-01T05:56:02.896 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.369574.log 2024-04-01T05:56:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.358095.log: /var/log/ceph/ceph-client.admin.219727.log 93.4% -- replaced with /var/log/ceph/ceph-client.admin.199304.log.gz 2024-04-01T05:56:02.898 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 96.4% -- replaced with /var/log/ceph/ceph-client.admin.549799.log.gz 2024-04-01T05:56:02.899 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose/var/log/ceph/ceph-client.admin.219727.log: -- /var/log/ceph/ceph-client.admin.391775.log 2024-04-01T05:56:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.369574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.256672.log 2024-04-01T05:56:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.391775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185795.log 2024-04-01T05:56:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.256672.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.185795.log: -- /var/log/ceph/ceph-client.admin.421801.log 2024-04-01T05:56:02.902 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.421801.log: /var/log/ceph/ceph-client.admin.483919.log 2024-04-01T05:56:02.903 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.483919.log: /var/log/ceph/ceph-client.admin.466416.log 2024-04-01T05:56:02.903 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137953.log 2024-04-01T05:56:02.903 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45536.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.466416.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45779.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45537.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 81.3% -- replaced with /var/log/ceph/ceph-client.admin.495249.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.543937.log 81.3% 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.496091.log.gz 2024-04-01T05:56:02.906 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.489555.log.gz 2024-04-01T05:56:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.543937.log: -- /var/log/ceph/ceph-client.admin.200194.log 2024-04-01T05:56:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.200194.log: /var/log/ceph/ceph-client.admin.556900.log 2024-04-01T05:56:02.911 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.489142.log.gz 2024-04-01T05:56:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.137953.log: --verbose --/var/log/ceph/ceph-client.admin.556900.log: /var/log/ceph/ceph-client.admin.47789.log 2024-04-01T05:56:02.911 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.556900.log.gz 2024-04-01T05:56:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47789.log: /var/log/ceph/ceph-client.admin.552019.log 2024-04-01T05:56:02.912 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.134563.log.gz 2024-04-01T05:56:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278087.log/var/log/ceph/ceph-client.admin.552019.log: 2024-04-01T05:56:02.912 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.552019.log.gz 2024-04-01T05:56:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.278087.log: /var/log/ceph/ceph-mgr.y.log 2024-04-01T05:56:02.914 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.137110.log.gz 2024-04-01T05:56:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.381461.log 2024-04-01T05:56:02.915 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196173.log 2024-04-01T05:56:02.915 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.381461.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.196173.log: /var/log/ceph/ceph-client.admin.237047.log 2024-04-01T05:56:02.915 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74011.log.gz 2024-04-01T05:56:02.915 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.237047.log: /var/log/ceph/ceph-client.admin.565289.log 2024-04-01T05:56:02.917 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.565289.log: /var/log/ceph/ceph-client.admin.51960.log 2024-04-01T05:56:02.917 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.565289.log.gz 2024-04-01T05:56:02.917 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-04-01T05:56:02.917 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451012.log 2024-04-01T05:56:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52128.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.451012.log: /var/log/ceph/ceph-client.admin.143167.log 2024-04-01T05:56:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.143167.log: /var/log/ceph/ceph-client.admin.537877.log 2024-04-01T05:56:02.920 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.524998.log.gz 2024-04-01T05:56:02.920 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.439029.log.gz 2024-04-01T05:56:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.537877.log: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.133717.log.gz 2024-04-01T05:56:02.920 INFO:teuthology.orchestra.run.smithi064.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.537877.log.gz 2024-04-01T05:56:02.921 INFO:teuthology.orchestra.run.smithi064.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.440724.log.gz 2024-04-01T05:56:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433050.log 2024-04-01T05:56:02.923 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.370426.log.gz 2024-04-01T05:56:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367879.log 2024-04-01T05:56:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.433050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.230067.log 2024-04-01T05:56:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.367879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140062.log 2024-04-01T05:56:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.230067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147537.log 2024-04-01T05:56:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.140062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52224.log 2024-04-01T05:56:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.147537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377060.log 2024-04-01T05:56:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120300.log 2024-04-01T05:56:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.377060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50222.log 2024-04-01T05:56:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.120300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.374653.log 2024-04-01T05:56:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50222.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.374653.log: /var/log/ceph/ceph-client.admin.201554.log 2024-04-01T05:56:02.927 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.463712.log.gz 2024-04-01T05:56:02.929 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.133301.log.gz 2024-04-01T05:56:02.929 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2024-04-01T05:56:02.929 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.439464.log.gz 2024-04-01T05:56:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.201554.log: /var/log/ceph/ceph-client.admin.266555.log 2024-04-01T05:56:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.266555.log: /var/log/ceph/ceph-client.admin.240475.log 2024-04-01T05:56:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.240475.log: /var/log/ceph/ceph-client.admin.552472.log 2024-04-01T05:56:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.552472.log: /var/log/ceph/ceph-client.admin.389828.log 2024-04-01T05:56:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51876.log 2024-04-01T05:56:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.554237.log 2024-04-01T05:56:02.935 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51876.log: /var/log/ceph/ceph-client.admin.389828.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.132032.log.gz 2024-04-01T05:56:02.935 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.552472.log.gz 2024-04-01T05:56:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138800.log 2024-04-01T05:56:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.554237.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.554237.log.gz 2024-04-01T05:56:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.537886.log 2024-04-01T05:56:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.138800.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.537886.log: /var/log/ceph/ceph-client.admin.213228.log 2024-04-01T05:56:02.938 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.138800.log.gz 2024-04-01T05:56:02.938 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.140062.log.gz 2024-04-01T05:56:02.939 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.537886.log.gz 2024-04-01T05:56:02.939 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51876.log.gz 2024-04-01T05:56:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.267818.log 2024-04-01T05:56:02.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.213228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51265.log 2024-04-01T05:56:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.267818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471211.log 2024-04-01T05:56:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428329.log 2024-04-01T05:56:02.945 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.471211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355049.log 2024-04-01T05:56:02.946 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.428329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47747.log 2024-04-01T05:56:02.947 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.355049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190282.log 2024-04-01T05:56:02.947 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469517.log 2024-04-01T05:56:02.948 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.190282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388917.log 2024-04-01T05:56:02.948 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.469517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-04-01T05:56:02.949 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.388917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355899.log 2024-04-01T05:56:02.949 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161719.log 2024-04-01T05:56:02.950 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.561812.log/var/log/ceph/ceph-client.admin.355899.log: /var/log/ceph/ceph-client.admin.161719.log: 2024-04-01T05:56:02.951 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.561812.log: /var/log/ceph/ceph-client.admin.368310.log 2024-04-01T05:56:02.952 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.232749.log 2024-04-01T05:56:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.368310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.507871.log 2024-04-01T05:56:02.954 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.232749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187128.log 2024-04-01T05:56:02.955 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0%/var/log/ceph/ceph-client.admin.507871.log: -- replaced with /var/log/ceph/ceph-client.admin.219306.log.gz 2024-04-01T05:56:02.956 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.221192.log.gz 2024-04-01T05:56:02.956 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155206.log 2024-04-01T05:56:02.957 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203796.log/var/log/ceph/ceph-client.admin.155206.log: 2024-04-01T05:56:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.187128.log: gzip 81.2% -- replaced with /var/log/ceph/ceph-client.admin.255829.log.gz 2024-04-01T05:56:02.959 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.483484.log 2024-04-01T05:56:02.959 INFO:teuthology.orchestra.run.smithi064.stderr: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.256250.log.gz 2024-04-01T05:56:02.960 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.479555.log.gz 2024-04-01T05:56:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.203796.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.483484.log: /var/log/ceph/ceph-client.admin.43953.log 2024-04-01T05:56:02.962 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.257975.log.gz 2024-04-01T05:56:02.962 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.126828.log.gz 2024-04-01T05:56:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358974.log/var/log/ceph/ceph-client.admin.43953.log: 2024-04-01T05:56:02.964 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.448443.log.gz 2024-04-01T05:56:02.964 INFO:teuthology.orchestra.run.smithi064.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.223349.log.gz 2024-04-01T05:56:02.964 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 81.4% -- replaced with /var/log/ceph/ceph-client.admin.449295.log.gz 2024-04-01T05:56:02.964 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.529217.log 2024-04-01T05:56:02.965 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155215.log 2024-04-01T05:56:02.965 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.358974.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.452122.log.gz 2024-04-01T05:56:02.966 INFO:teuthology.orchestra.run.smithi064.stderr: 82.9% -- replaced with /var/log/ceph/ceph-client.admin.476108.log.gz 2024-04-01T05:56:02.966 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.465433.log.gz 2024-04-01T05:56:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.529217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433059.log 2024-04-01T05:56:02.968 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.466416.log.gz 2024-04-01T05:56:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.155215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.464133.log 2024-04-01T05:56:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.433059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-04-01T05:56:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.464133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51560.log 2024-04-01T05:56:02.970 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.342643.log.gz 2024-04-01T05:56:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45203.log: -- /var/log/ceph/ceph-client.admin.199295.log 2024-04-01T05:56:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.199295.log: /var/log/ceph/ceph-client.admin.49340.log 2024-04-01T05:56:02.973 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.199754.log 2024-04-01T05:56:02.973 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.381461.log.gz 2024-04-01T05:56:02.974 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45195.log 2024-04-01T05:56:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.199754.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.355049.log.gz 2024-04-01T05:56:02.976 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% 81.0% -- replaced with /var/log/ceph/ceph-client.admin.278087.log.gz 2024-04-01T05:56:02.976 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.524286.log.gz 2024-04-01T05:56:02.976 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.234890.log 2024-04-01T05:56:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45195.log: /var/log/ceph/ceph-client.admin.382748.log 2024-04-01T05:56:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536109.log 2024-04-01T05:56:02.981 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.382748.log: /var/log/ceph/ceph-client.admin.234890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.352805.log 2024-04-01T05:56:02.981 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.453394.log.gz 2024-04-01T05:56:02.982 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.536109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51476.log 2024-04-01T05:56:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.352805.log: /var/log/ceph/ceph-client.admin.342634.log 2024-04-01T05:56:02.984 INFO:teuthology.orchestra.run.smithi064.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.374653.log.gz 2024-04-01T05:56:02.984 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.536109.log.gz 2024-04-01T05:56:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-04-01T05:56:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51476.log: /var/log/ceph/ceph-client.admin.342634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.535195.log 90.2% -- replaced with /var/log/ceph/ceph-client.admin.211546.log.gz 2024-04-01T05:56:02.986 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:02.987 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44813.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31314.log 2024-04-01T05:56:02.988 INFO:teuthology.orchestra.run.smithi064.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.348984.log.gz 2024-04-01T05:56:02.988 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.538719.log 2024-04-01T05:56:02.988 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.253712.log.gz 2024-04-01T05:56:02.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.535195.log: /var/log/ceph/ceph.tmp-client.admin.31314.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31314.log.gz 2024-04-01T05:56:02.990 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.507459.log 2024-04-01T05:56:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.538719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243586.log 2024-04-01T05:56:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.248243.log 2024-04-01T05:56:02.992 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3%/var/log/ceph/ceph-client.admin.243586.log: -- replaced with /var/log/ceph/ceph-client.admin.507871.log.gz 2024-04-01T05:56:02.992 INFO:teuthology.orchestra.run.smithi064.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.505665.log.gz 2024-04-01T05:56:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.507459.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.499975.log.gz 2024-04-01T05:56:02.994 INFO:teuthology.orchestra.run.smithi064.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.538719.log.gz 2024-04-01T05:56:02.994 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.502110.log.gz 2024-04-01T05:56:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.425757.log 2024-04-01T05:56:02.994 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.229129.log.gz 2024-04-01T05:56:02.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205607.log 2024-04-01T05:56:02.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.248243.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.433050.log.gz 2024-04-01T05:56:03.000 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.433059.log.gz 2024-04-01T05:56:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.425757.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.436905.log.gz 2024-04-01T05:56:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 90.1% -- replaced with /var/log/ceph/ceph-client.admin.252782.log.gz 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% 81.4% -- replaced with /var/log/ceph/ceph-client.admin.185795.log.gz -- replaced with /var/log/ceph/ceph-client.admin.185361.log.gz 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.476950.log 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.205607.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.367879.log.gz 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.501683.log 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.476950.log: 80.9%gzip -5 78.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.152614.log.gz 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.545016.log/var/log/ceph/ceph-client.admin.501683.log: 2024-04-01T05:56:03.001 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.425757.log.gz 2024-04-01T05:56:03.002 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44981.log.gz 2024-04-01T05:56:03.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.244490.log 2024-04-01T05:56:03.003 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.382748.log.gz 2024-04-01T05:56:03.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.545016.log: /var/log/ceph/ceph-client.admin.473875.log 2024-04-01T05:56:03.003 INFO:teuthology.orchestra.run.smithi064.stderr: 84.4% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-04-01T05:56:03.004 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.545016.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451877.log 2024-04-01T05:56:03.004 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.244490.log: /var/log/ceph/ceph-client.admin.473875.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.228686.log 2024-04-01T05:56:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.451877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49340.log.gz 2024-04-01T05:56:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112721.log 2024-04-01T05:56:03.006 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49256.log.gz 2024-04-01T05:56:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.228686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240484.log 2024-04-01T05:56:03.007 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2%/var/log/ceph/ceph-client.admin.112721.log: -- replaced with /var/log/ceph/ceph-client.admin.412777.log.gz 2024-04-01T05:56:03.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467751.log 2024-04-01T05:56:03.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.240484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163081.log 2024-04-01T05:56:03.011 INFO:teuthology.orchestra.run.smithi064.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.417511.log.gz 2024-04-01T05:56:03.011 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.414942.log.gz 2024-04-01T05:56:03.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.163081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.447611.log 2024-04-01T05:56:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.467751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.555586.log 2024-04-01T05:56:03.012 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.419599.log.gz 2024-04-01T05:56:03.013 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.447611.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.420495.log.gz 2024-04-01T05:56:03.013 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142753.log 2024-04-01T05:56:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51434.log.gz 2024-04-01T05:56:03.014 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.209008.log 2024-04-01T05:56:03.015 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203345.log 2024-04-01T05:56:03.016 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.135827.log.gz 2024-04-01T05:56:03.016 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.476950.log.gz 2024-04-01T05:56:03.017 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0%/var/log/ceph/ceph-client.admin.555586.log: -- replaced with /var/log/ceph/ceph-client.admin.231063.log.gz 2024-04-01T05:56:03.017 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.529217.log.gz 2024-04-01T05:56:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.209008.log: /var/log/ceph/ceph-client.admin.142753.log: 87.4% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45198.log.gz 2024-04-01T05:56:03.019 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45195.log.gz 2024-04-01T05:56:03.020 INFO:teuthology.orchestra.run.smithi064.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.342634.log.gz 2024-04-01T05:56:03.020 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.201554.log.gz 2024-04-01T05:56:03.021 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.152623.log.gz 2024-04-01T05:56:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.203345.log: /var/log/ceph/ceph-client.admin.530068.log 2024-04-01T05:56:03.023 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.209008.log.gz 2024-04-01T05:56:03.023 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.447611.log.gz 2024-04-01T05:56:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.530068.log: /var/log/ceph/ceph-client.admin.126397.log 2024-04-01T05:56:03.025 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.126397.log: /var/log/ceph/ceph-client.admin.182215.log 2024-04-01T05:56:03.026 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.244490.log.gz 90.2% -- replaced with /var/log/ceph/ceph-client.admin.248243.log.gz 2024-04-01T05:56:03.026 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.027 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188028.log 2024-04-01T05:56:03.027 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.182215.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.380991.log.gz 2024-04-01T05:56:03.028 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180940.log 2024-04-01T05:56:03.029 INFO:teuthology.orchestra.run.smithi064.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.555586.log.gz 2024-04-01T05:56:03.029 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.188028.log: /var/log/ceph/ceph-client.admin.47705.log 2024-04-01T05:56:03.030 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.352805.log.gz 2024-04-01T05:56:03.030 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.252773.log 2024-04-01T05:56:03.031 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.180940.log: /var/log/ceph/ceph-client.admin.47705.log: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.155206.log.gz 2024-04-01T05:56:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.246493.log 2024-04-01T05:56:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.252773.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.251939.log.gz 2024-04-01T05:56:03.033 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.155215.log.gz 2024-04-01T05:56:03.033 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.203805.log 2024-04-01T05:56:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.246493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.553359.log 2024-04-01T05:56:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.203805.log: gzip 78.8% -- replaced with /var/log/ceph/ceph-client.admin.282536.log.gz 2024-04-01T05:56:03.036 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.439884.log 2024-04-01T05:56:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.553359.log: 80.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.202445.log 2024-04-01T05:56:03.037 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.279397.log.gz 2024-04-01T05:56:03.038 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.498708.log.gz 2024-04-01T05:56:03.038 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.499553.log.gz 2024-04-01T05:56:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.202445.log: /var/log/ceph/ceph-client.admin.439884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130684.log 2024-04-01T05:56:03.040 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.504686.log 2024-04-01T05:56:03.040 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.530068.log.gz 2024-04-01T05:56:03.040 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.238305.log.gz 2024-04-01T05:56:03.041 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.527539.log.gz 2024-04-01T05:56:03.042 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.130684.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.413206.log.gzgzip 2024-04-01T05:56:03.042 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.247821.log 2024-04-01T05:56:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.504686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.557343.log 2024-04-01T05:56:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.473445.log 2024-04-01T05:56:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.247821.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.200194.log.gz 2024-04-01T05:56:03.044 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.199295.log.gz 2024-04-01T05:56:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.557343.log: gzip 80.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.421379.log 2024-04-01T05:56:03.046 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.557343.log.gz 2024-04-01T05:56:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.473445.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.421379.log: /var/log/ceph/ceph-client.admin.368739.log 2024-04-01T05:56:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 81.0% -- replaced with /var/log/ceph/ceph-client.admin.471202.log.gz 2024-04-01T05:56:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.368739.log: -- /var/log/ceph/ceph-client.admin.513750.log 2024-04-01T05:56:03.047 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.553359.log.gz 2024-04-01T05:56:03.048 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506369.log 2024-04-01T05:56:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.513750.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.237047.log.gz 2024-04-01T05:56:03.050 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- 90.1% -- replaced with /var/log/ceph/ceph-client.admin.473875.log.gz 2024-04-01T05:56:03.050 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.506369.log: /var/log/ceph/ceph-client.admin.362755.log 2024-04-01T05:56:03.051 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.501683.log.gz 2024-04-01T05:56:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548064.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-04-01T05:56:03.052 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.053 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.281275.log.gz 2024-04-01T05:56:03.053 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.368739.log.gz 2024-04-01T05:56:03.053 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.366622.log.gz 79.2% 2024-04-01T05:56:03.053 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.417933.log.gz 2024-04-01T05:56:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197982.log 2024-04-01T05:56:03.055 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2024-04-01T05:56:03.055 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2024-04-01T05:56:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.548064.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.548064.log.gz 2024-04-01T05:56:03.056 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1%gzip -5/var/log/ceph/ceph-client.admin.197982.log: --verbose -- /var/log/ceph/ceph-client.admin.258818.log/var/log/ceph/ceph-client.admin.362755.log: 2024-04-01T05:56:03.056 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2024-04-01T05:56:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114642.log 2024-04-01T05:56:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.258818.log: gzip -5 81.2% -- replaced with /var/log/ceph/ceph-client.admin.231906.log.gz 2024-04-01T05:56:03.059 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.370851.log 2024-04-01T05:56:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2024-04-01T05:56:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.370851.log: -- /var/log/ceph/ceph-client.admin.465009.log 2024-04-01T05:56:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-04-01T05:56:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.465009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.345173.log 2024-04-01T05:56:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.469943.log 2024-04-01T05:56:03.067 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.345173.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.241775.log 2024-04-01T05:56:03.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.469943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.556012.log 2024-04-01T05:56:03.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.241775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192548.log 2024-04-01T05:56:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.556012.log: 90.1% 90.1% 80.9% -- replaced with /var/log/ceph/ceph-client.admin.556012.log.gz 2024-04-01T05:56:03.072 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.423934.log.gz 2024-04-01T05:56:03.072 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.247821.log.gz 2024-04-01T05:56:03.072 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.361915.log.gz 2024-04-01T05:56:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.390273.log 2024-04-01T05:56:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.192548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.377545.log 2024-04-01T05:56:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-01T05:56:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.390273.log: /var/log/ceph/ceph-client.admin.377545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.546310.log 2024-04-01T05:56:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.524577.log 2024-04-01T05:56:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.546310.log: /var/log/ceph/ceph-osd.0.log: gzip 96.6% -- replaced with /var/log/ceph/ceph-client.admin.539651.log.gz 2024-04-01T05:56:03.078 INFO:teuthology.orchestra.run.smithi064.stderr: 77.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.132445.log 2024-04-01T05:56:03.078 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.546310.log.gz 2024-04-01T05:56:03.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.278959.log 2024-04-01T05:56:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.132445.log: /var/log/ceph/ceph-client.admin.524577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.528795.log 2024-04-01T05:56:03.081 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.278959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384853.log 2024-04-01T05:56:03.082 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.464133.log.gz 2024-04-01T05:56:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.528795.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48209.log.gz 2024-04-01T05:56:03.083 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.255838.log 2024-04-01T05:56:03.084 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126388.log 2024-04-01T05:56:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.255838.log: /var/log/ceph/ceph-client.admin.386191.log 2024-04-01T05:56:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.126388.log: /var/log/ceph/ceph-client.admin.384853.log: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.473445.log.gz 2024-04-01T05:56:03.086 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.216751.log.gz 2024-04-01T05:56:03.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.356832.log 2024-04-01T05:56:03.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.386191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.246939.log/var/log/ceph/ceph-client.admin.356832.log: 2024-04-01T05:56:03.089 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485712.log 2024-04-01T05:56:03.089 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.236617.log.gz 2024-04-01T05:56:03.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.246939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.395684.log 2024-04-01T05:56:03.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361066.log 2024-04-01T05:56:03.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.395684.log: /var/log/ceph/ceph-client.admin.361066.log: 80.7%/var/log/ceph/ceph-client.admin.485712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.237470.log 2024-04-01T05:56:03.093 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.528795.log.gz 2024-04-01T05:56:03.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.237470.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.272104.log 2024-04-01T05:56:03.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.272104.log: /var/log/ceph/ceph-client.admin.211537.log 2024-04-01T05:56:03.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.211537.log: /var/log/ceph/ceph-client.admin.477380.log 2024-04-01T05:56:03.096 INFO:teuthology.orchestra.run.smithi064.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.451765.log.gz 2024-04-01T05:56:03.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467325.log 2024-04-01T05:56:03.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.477380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214086.log 2024-04-01T05:56:03.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.467325.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.214086.log: /var/log/ceph/ceph-client.admin.550252.log 2024-04-01T05:56:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 81.1% -- replaced with /var/log/ceph/ceph-client.admin.506369.log.gz 2024-04-01T05:56:03.099 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.358541.log 2024-04-01T05:56:03.100 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.358541.log: /var/log/ceph/ceph-client.admin.48479.log 2024-04-01T05:56:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 77.5% -- replaced with /var/log/ceph/ceph-client.admin.203345.log.gz 2024-04-01T05:56:03.102 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- 76.7% -- replaced with /var/log/ceph/ceph-client.admin.202445.log.gz 2024-04-01T05:56:03.102 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.269091.log 2024-04-01T05:56:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48479.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.214977.log.gz 2024-04-01T05:56:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.550252.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.349405.log.gz 2024-04-01T05:56:03.103 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.192548.log.gz 2024-04-01T05:56:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.375106.log 2024-04-01T05:56:03.104 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.182646.log.gz 2024-04-01T05:56:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.269091.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.182215.log.gz 2024-04-01T05:56:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564291.log 2024-04-01T05:56:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.375106.log: --verbose/var/log/ceph/ceph-client.admin.564291.log: -- /var/log/ceph/ceph-client.admin.509167.log 2024-04-01T05:56:03.107 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.564291.log.gz 2024-04-01T05:56:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.525841.log 2024-04-01T05:56:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.509167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409616.log 2024-04-01T05:56:03.110 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.550252.log.gz 2024-04-01T05:56:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.408715.log 2024-04-01T05:56:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.525841.log: gzip -5/var/log/ceph/ceph-client.admin.409616.log: /var/log/ceph/ceph-client.admin.408715.log: --verbose -- /var/log/ceph/ceph-client.admin.345594.log 2024-04-01T05:56:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225885.log 2024-04-01T05:56:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.345594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.237892.log 2024-04-01T05:56:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.225885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451756.log 2024-04-01T05:56:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.237892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.262214.log 2024-04-01T05:56:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.262214.log: /var/log/ceph/ceph-client.admin.529647.log 2024-04-01T05:56:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.451756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.559183.log 2024-04-01T05:56:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.559183.log: /var/log/ceph/ceph-client.admin.240026.log 2024-04-01T05:56:03.119 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% 81.1% -- replaced with /var/log/ceph/ceph-client.admin.559183.log.gz 2024-04-01T05:56:03.119 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.129421.log.gz 2024-04-01T05:56:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.552916.log 2024-04-01T05:56:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.240026.log: /var/log/ceph/ceph-client.admin.529647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.360646.log 2024-04-01T05:56:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.532599.log 2024-04-01T05:56:03.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.360646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136689.log 2024-04-01T05:56:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.552916.log: /var/log/ceph/ceph-client.admin.532599.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.357244.log.gz 2024-04-01T05:56:03.123 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.356832.log.gz 2024-04-01T05:56:03.124 INFO:teuthology.orchestra.run.smithi064.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.184457.log.gz 2024-04-01T05:56:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-04-01T05:56:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.136689.log: /var/log/ceph/ceph-client.admin.48784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388908.log 2024-04-01T05:56:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380121.log 2024-04-01T05:56:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.388908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50138.log 2024-04-01T05:56:03.127 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.188028.log.gz 2024-04-01T05:56:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.380121.log: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.241331.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.221604.log 2024-04-01T05:56:03.128 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50138.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.252773.log.gz 2024-04-01T05:56:03.129 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.240484.log.gz 78.1% 2024-04-01T05:56:03.129 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.532599.log.gz 2024-04-01T05:56:03.129 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.533021.log.gz 2024-04-01T05:56:03.130 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.532608.log.gz 2024-04-01T05:56:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 83.8% -- replaced with /var/log/ceph/ceph-client.admin.240026.log.gz 2024-04-01T05:56:03.131 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- 83.2% -- replaced with /var/log/ceph/ceph-client.admin.524577.log.gz 2024-04-01T05:56:03.131 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.393771.log 2024-04-01T05:56:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.393771.log: /var/log/ceph/ceph-client.admin.369162.log 2024-04-01T05:56:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.221604.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.451886.log.gz 2024-04-01T05:56:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.530913.log 2024-04-01T05:56:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.369162.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.552916.log.gz 2024-04-01T05:56:03.135 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.545878.log 2024-04-01T05:56:03.135 INFO:teuthology.orchestra.run.smithi064.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.210693.log.gz 2024-04-01T05:56:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.197523.log 2024-04-01T05:56:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.530913.log: /var/log/ceph/ceph-client.admin.545878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.479132.log 2024-04-01T05:56:03.138 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.346872.log.gz 2024-04-01T05:56:03.138 INFO:teuthology.orchestra.run.smithi064.stderr: 79.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.545878.log.gz 2024-04-01T05:56:03.138 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.479132.log: /var/log/ceph/ceph-client.admin.52044.log 2024-04-01T05:56:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52044.log: /var/log/ceph/ceph-client.admin.554689.log 2024-04-01T05:56:03.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.197523.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.363588.log.gz 2024-04-01T05:56:03.142 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.485712.log.gz 2024-04-01T05:56:03.142 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2%/var/log/ceph/ceph-client.admin.554689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.392297.log 2024-04-01T05:56:03.143 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.179661.log.gz 2024-04-01T05:56:03.143 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.369162.log.gz 2024-04-01T05:56:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490888.log 2024-04-01T05:56:03.144 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gz 2024-04-01T05:56:03.145 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47831.log.gz 2024-04-01T05:56:03.145 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.146 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.554689.log.gz 2024-04-01T05:56:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 90.1% -- replaced with /var/log/ceph/ceph-client.admin.377545.log.gz 2024-04-01T05:56:03.147 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose/var/log/ceph/ceph-client.admin.490888.log: -- /var/log/ceph/ceph-client.admin.156532.log 2024-04-01T05:56:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.392297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51690.log 2024-04-01T05:56:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51690.log: /var/log/ceph/ceph-client.admin.52266.log/var/log/ceph/ceph-client.admin.156532.log: 2024-04-01T05:56:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.494316.log 2024-04-01T05:56:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-04-01T05:56:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.558721.log 2024-04-01T05:56:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45737.log: /var/log/ceph/ceph-client.admin.494316.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.558721.log: /var/log/ceph/ceph-client.admin.383169.log 2024-04-01T05:56:03.153 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.558721.log.gz 2024-04-01T05:56:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.499132.log 2024-04-01T05:56:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.383169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.279406.log 2024-04-01T05:56:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.499132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144482.log 2024-04-01T05:56:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.279406.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50138.log.gz 2024-04-01T05:56:03.158 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.351532.log 2024-04-01T05:56:03.159 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.368730.log 2024-04-01T05:56:03.159 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50894.log.gz 2024-04-01T05:56:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.144482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.527108.log 2024-04-01T05:56:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443283.log 2024-04-01T05:56:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.351532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541013.log 2024-04-01T05:56:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.443283.log: /var/log/ceph/ceph-client.admin.527108.log: /var/log/ceph/ceph-client.admin.368730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.205147.log 2024-04-01T05:56:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.541013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.468219.log 2024-04-01T05:56:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.205147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215427.log 2024-04-01T05:56:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380112.log 2024-04-01T05:56:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.215427.log: /var/log/ceph/ceph-client.admin.468219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-04-01T05:56:03.170 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.136689.log.gz 2024-04-01T05:56:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.380112.log: gzip -5 91.3% -- replaced with /var/log/ceph/ceph-client.admin.541013.log.gz 2024-04-01T05:56:03.171 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.504695.log 2024-04-01T05:56:03.173 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.443292.log 2024-04-01T05:56:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.504695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483035.log 2024-04-01T05:56:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.443292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.512504.log 2024-04-01T05:56:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.483035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384440.log 2024-04-01T05:56:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.512504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45989.log 2024-04-01T05:56:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.544162.log 2024-04-01T05:56:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.384440.log: /var/log/ceph/ceph-client.admin.45989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.427903.log 2024-04-01T05:56:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.544162.log: gzip -5 --verbose -- 84.2% /var/log/ceph/ceph-client.admin.44603.log 2024-04-01T05:56:03.180 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.544162.log.gz 2024-04-01T05:56:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.427903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.243595.log 81.9% -- replaced with /var/log/ceph/ceph-client.admin.217203.log.gz 2024-04-01T05:56:03.182 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.479132.log.gz 2024-04-01T05:56:03.182 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.182 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.221604.log.gz 2024-04-01T05:56:03.183 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.362326.log 2024-04-01T05:56:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.243595.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-01T05:56:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.362326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52350.log 2024-04-01T05:56:03.186 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.540560.log.gz 2024-04-01T05:56:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose 78.3% -- replaced with /var/log/ceph/ceph-client.admin.265292.log.gz 2024-04-01T05:56:03.187 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.564375.log 2024-04-01T05:56:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.344325.log 2024-04-01T05:56:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48041.log 2024-04-01T05:56:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.344325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565668.log 2024-04-01T05:56:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564375.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.48041.log: /var/log/ceph/ceph-client.admin.565668.log: -- /var/log/ceph/ceph-client.admin.431787.log 2024-04-01T05:56:03.191 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.565668.log.gz 2024-04-01T05:56:03.191 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.564375.log.gz 2024-04-01T05:56:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147963.log 2024-04-01T05:56:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.431787.log: -5 --verbose --/var/log/ceph/ceph-client.admin.147963.log: /var/log/ceph/ceph-client.admin.280843.log 2024-04-01T05:56:03.193 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.411334.log 2024-04-01T05:56:03.193 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.467325.log.gz 2024-04-01T05:56:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.411334.log: /var/log/ceph/ceph-client.admin.555124.log 2024-04-01T05:56:03.195 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.214527.log.gz 2024-04-01T05:56:03.195 INFO:teuthology.orchestra.run.smithi064.stderr: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.215877.log.gz 2024-04-01T05:56:03.195 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.345594.log.gz 2024-04-01T05:56:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.555124.log: /var/log/ceph/ceph-client.admin.160315.log 2024-04-01T05:56:03.197 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.350251.log.gz 2024-04-01T05:56:03.198 INFO:teuthology.orchestra.run.smithi064.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.555124.log.gz 2024-04-01T05:56:03.198 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.468219.log.gz 2024-04-01T05:56:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.280843.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.344756.log.gz 2024-04-01T05:56:03.198 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.215427.log.gz 2024-04-01T05:56:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.354637.log 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.160315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147084.log 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.477792.log 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.147084.log: 90.3%/var/log/ceph/ceph-client.admin.354637.log: -- replaced with /var/log/ceph/ceph-client.admin.206912.log.gz 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.367888.log 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.477792.log: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.534312.log.gz 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47663.log 2024-04-01T05:56:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.367888.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.534754.log.gz 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 81.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.380112.log.gz 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.426618.log 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.380121.log.gz 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr: 79.7% -- replaced with /var/log/ceph/ceph-client.admin.241766.log.gz 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47663.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.377966.log.gz 2024-04-01T05:56:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462018.log 2024-04-01T05:56:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163493.log 2024-04-01T05:56:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.426618.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.163493.log: -- /var/log/ceph/ceph-client.admin.502943.log 2024-04-01T05:56:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.502943.log: -- /var/log/ceph/ceph-client.admin.538728.log 2024-04-01T05:56:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.538728.log: -- /var/log/ceph/ceph-client.admin.220148.log 2024-04-01T05:56:03.210 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.526272.log.gz 2024-04-01T05:56:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.462018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116947.log 2024-04-01T05:56:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.220148.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.538728.log.gz 2024-04-01T05:56:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144065.log 2024-04-01T05:56:03.213 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.490473.log.gz 2024-04-01T05:56:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.144065.log: /var/log/ceph/ceph-client.admin.51918.log/var/log/ceph/ceph-client.admin.116947.log: 2024-04-01T05:56:03.214 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.490888.log.gz 2024-04-01T05:56:03.215 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.492995.log.gz 2024-04-01T05:56:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179234.log 2024-04-01T05:56:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51918.log: --/var/log/ceph/ceph-client.admin.179234.log: /var/log/ceph/ceph-osd.2.log 2024-04-01T05:56:03.216 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.530913.log.gz 2024-04-01T05:56:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.269100.log 2024-04-01T05:56:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose 90.0% -- replaced with /var/log/ceph/ceph-client.admin.531344.log.gz 2024-04-01T05:56:03.218 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.136256.log 2024-04-01T05:56:03.219 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.439884.log.gz 2024-04-01T05:56:03.219 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-04-01T05:56:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.269100.log: /var/log/ceph/ceph-client.admin.136256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476529.log 2024-04-01T05:56:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.476529.log: /var/log/ceph/ceph-client.admin.387147.log 2024-04-01T05:56:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489564.log 2024-04-01T05:56:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.387147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.252360.log 2024-04-01T05:56:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489564.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48479.log.gz 2024-04-01T05:56:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-04-01T05:56:03.225 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48563.log.gz 2024-04-01T05:56:03.226 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.235723.log 2024-04-01T05:56:03.226 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.197523.log.gz 2024-04-01T05:56:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.252360.log: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.280843.log.gz 2024-04-01T05:56:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226315.log 2024-04-01T05:56:03.228 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.389407.log.gz 2024-04-01T05:56:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.235723.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.431787.log.gz 2024-04-01T05:56:03.228 INFO:teuthology.orchestra.run.smithi064.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.361066.log.gz 2024-04-01T05:56:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.212383.log 2024-04-01T05:56:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.433903.log 2024-04-01T05:56:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.212383.log: gzip -5 90.1% --verbose -- /var/log/ceph/ceph-client.admin.360226.log 2024-04-01T05:56:03.231 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.433903.log: -- replaced with /var/log/ceph/ceph-client.admin.426618.log.gz 2024-04-01T05:56:03.231 INFO:teuthology.orchestra.run.smithi064.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.410901.log.gz 2024-04-01T05:56:03.231 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.226315.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.153471.log.gz 2024-04-01T05:56:03.232 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.429649.log 2024-04-01T05:56:03.232 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2024-04-01T05:56:03.232 INFO:teuthology.orchestra.run.smithi064.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.116447.log.gz 2024-04-01T05:56:03.233 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.429649.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44477.log: /var/log/ceph/ceph-mds.c.log 85.6% -- replaced with /var/log/ceph/ceph-client.admin.126388.log.gz 2024-04-01T05:56:03.234 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.234 INFO:teuthology.orchestra.run.smithi064.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.116938.log.gz 2024-04-01T05:56:03.235 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.360226.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-04-01T05:56:03.235 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2024-04-01T05:56:03.235 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49214.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 2024-04-01T05:56:03.237 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.422682.log 2024-04-01T05:56:03.237 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-04-01T05:56:03.238 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: 2024-04-01T05:56:03.238 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.540106.log.gz 2024-04-01T05:56:03.238 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.351532.log.gz 2024-04-01T05:56:03.238 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.453815.log 2024-04-01T05:56:03.240 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.445625.log 2024-04-01T05:56:03.240 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.241 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.453815.log: /var/log/ceph/ceph-client.admin.422682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.515055.log 2024-04-01T05:56:03.241 INFO:teuthology.orchestra.run.smithi064.stderr: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.183957.log.gz 2024-04-01T05:56:03.241 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.445625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.482436.log 2024-04-01T05:56:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.515055.log: /var/log/ceph/ceph-client.admin.125645.log 2024-04-01T05:56:03.242 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.130254.log.gz 2024-04-01T05:56:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.253194.log 2024-04-01T05:56:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.482436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.215886.log 2024-04-01T05:56:03.244 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.125645.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.253194.log: /var/log/ceph/ceph-client.admin.452974.log 2024-04-01T05:56:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.493464.log 2024-04-01T05:56:03.246 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49808.log.gz 87.6% 2024-04-01T05:56:03.246 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-04-01T05:56:03.246 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.422682.log.gz 2024-04-01T05:56:03.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.215886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.472054.log 2024-04-01T05:56:03.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.452974.log: /var/log/ceph/ceph-client.admin.493464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188895.log 2024-04-01T05:56:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.268672.log 2024-04-01T05:56:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.472054.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.160324.log.gz 2024-04-01T05:56:03.249 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph.log.gzgzip 2024-04-01T05:56:03.249 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115495.log 2024-04-01T05:56:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.188895.log: /var/log/ceph/ceph-client.admin.268672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.448452.log 2024-04-01T05:56:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44034.log 2024-04-01T05:56:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115495.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.448452.log: -- /var/log/ceph/ceph-client.admin.560500.log 2024-04-01T05:56:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.564501.log 2024-04-01T05:56:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.461549.log 2024-04-01T05:56:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.560500.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.560500.log.gz 2024-04-01T05:56:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.564501.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.144065.log.gz 2024-04-01T05:56:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.487529.log 2024-04-01T05:56:03.255 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.118548.log.gz 2024-04-01T05:56:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.461549.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.564501.log.gz 2024-04-01T05:56:03.256 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.474788.log 2024-04-01T05:56:03.256 INFO:teuthology.orchestra.run.smithi064.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.386662.log.gz 2024-04-01T05:56:03.256 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.477792.log.gz 2024-04-01T05:56:03.256 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.217212.log.gz 2024-04-01T05:56:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.487529.log: gzip -5 --verbose -- 90.2% /var/log/ceph/ceph-client.admin.502530.log -- replaced with /var/log/ceph/ceph-client.admin.463291.log.gz 2024-04-01T05:56:03.258 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.219315.log.gz 2024-04-01T05:56:03.258 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.474788.log: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.181353.log.gz 2024-04-01T05:56:03.260 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.452974.log.gz 2024-04-01T05:56:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50474.log 2024-04-01T05:56:03.261 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.243173.log.gz 2024-04-01T05:56:03.261 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 90.1% -- replaced with /var/log/ceph/ceph-client.admin.180088.log.gz 2024-04-01T05:56:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.282545.log 2024-04-01T05:56:03.262 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.257107.log.gz 2024-04-01T05:56:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.502530.log: /var/log/ceph/ceph-client.admin.50474.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.209438.log.gz 2024-04-01T05:56:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180076.log 2024-04-01T05:56:03.265 INFO:teuthology.orchestra.run.smithi064.stderr: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.228226.log.gz 2024-04-01T05:56:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.282545.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.479564.log 2024-04-01T05:56:03.265 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1%/var/log/ceph/ceph-client.admin.180076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208596.log 2024-04-01T05:56:03.266 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.252360.log.gz 2024-04-01T05:56:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.479564.log: /var/log/ceph/ceph-client.admin.208596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348558.log 2024-04-01T05:56:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220157.log 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.226315.log.gz 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.269100.log.gz 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.348558.log: gzip -5 --verbose 84.1% -- replaced with /var/log/ceph/ceph-client.admin.487529.log.gz 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr: -- 90.0% -- replaced with /var/log/ceph/ceph-client.admin.247394.log.gz 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.251508.log 2024-04-01T05:56:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.220157.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.489564.log.gz 2024-04-01T05:56:03.271 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.435600.log 2024-04-01T05:56:03.271 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.251508.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.406924.log.gz 2024-04-01T05:56:03.272 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.357665.log 2024-04-01T05:56:03.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.435600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.561803.log 2024-04-01T05:56:03.272 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-04-01T05:56:03.273 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.357665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.513760.log 2024-04-01T05:56:03.273 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-04-01T05:56:03.273 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.561803.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.561803.log.gz 2024-04-01T05:56:03.273 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227185.log 2024-04-01T05:56:03.274 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.513760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.382310.log 2024-04-01T05:56:03.275 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.545450.log 2024-04-01T05:56:03.275 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.227185.log: /var/log/ceph/ceph-client.admin.382310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378387.log 2024-04-01T05:56:03.276 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.467316.log 2024-04-01T05:56:03.277 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.545450.log: /var/log/ceph/ceph-client.admin.378387.log: gzip -5 --verbose 79.2% -- replaced with /var/log/ceph/ceph-client.admin.545450.log.gz 2024-04-01T05:56:03.277 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.534745.log 2024-04-01T05:56:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.467316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164487.log 2024-04-01T05:56:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.534745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.545889.log 2024-04-01T05:56:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.164487.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.545889.log: /var/log/ceph/ceph-client.admin.430949.log 2024-04-01T05:56:03.281 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.545889.log.gz 2024-04-01T05:56:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260092.log 2024-04-01T05:56:03.284 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.430949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.258809.log 2024-04-01T05:56:03.285 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.260092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155627.log 2024-04-01T05:56:03.285 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.258809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-04-01T05:56:03.287 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.155627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565415.log 2024-04-01T05:56:03.288 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.224623.log 2024-04-01T05:56:03.289 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2%/var/log/ceph/ceph-client.admin.565415.log: -- replaced with /var/log/ceph/ceph-client.admin.44034.log.gz 2024-04-01T05:56:03.289 INFO:teuthology.orchestra.run.smithi064.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.351103.log.gz 2024-04-01T05:56:03.290 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.462448.log 2024-04-01T05:56:03.290 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.565415.log.gz 2024-04-01T05:56:03.291 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.224623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231493.log 2024-04-01T05:56:03.291 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.462448.log: gzip 80.9% -- replaced with /var/log/ceph/ceph-client.admin.231484.log.gz 2024-04-01T05:56:03.292 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.347284.log 2024-04-01T05:56:03.292 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-04-01T05:56:03.293 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.231493.log: /var/log/ceph/ceph-client.admin.347284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548919.log 2024-04-01T05:56:03.294 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.394229.log 2024-04-01T05:56:03.296 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.548919.log: gzip -5 --verbose -- 84.9% -- replaced with /var/log/ceph/ceph-client.admin.548919.log.gz 2024-04-01T05:56:03.296 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.414489.log 2024-04-01T05:56:03.297 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.394229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.264871.log 2024-04-01T05:56:03.298 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.414489.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.435600.log.gz 2024-04-01T05:56:03.298 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-04-01T05:56:03.299 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.264871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186255.log 2024-04-01T05:56:03.299 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.471633.log 2024-04-01T05:56:03.300 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.186255.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.433903.log.gz 2024-04-01T05:56:03.300 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.261362.log 2024-04-01T05:56:03.301 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385274.log 2024-04-01T05:56:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.471633.log: /var/log/ceph/ceph-client.admin.261362.log: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.388047.log.gz 2024-04-01T05:56:03.303 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.282545.log.gz 2024-04-01T05:56:03.305 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.508311.log 2024-04-01T05:56:03.305 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.199754.log.gz 2024-04-01T05:56:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.385274.log: -- replaced with /var/log/ceph/ceph-client.admin.365299.log.gz 2024-04-01T05:56:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388495.log 2024-04-01T05:56:03.308 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.508311.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.235732.log.gz 2024-04-01T05:56:03.308 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451021.log 2024-04-01T05:56:03.308 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.238736.log.gz 2024-04-01T05:56:03.308 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.364439.log 2024-04-01T05:56:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.388495.log: /var/log/ceph/ceph-client.admin.451021.log: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.360226.log.gz 2024-04-01T05:56:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.385715.log 2024-04-01T05:56:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.364439.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.473866.log 2024-04-01T05:56:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.385715.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.362746.log.gz 2024-04-01T05:56:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137944.log 2024-04-01T05:56:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.222876.log/var/log/ceph/ceph-client.admin.473866.log: 2024-04-01T05:56:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.137944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-04-01T05:56:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.426183.log 2024-04-01T05:56:03.315 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51774.log.gz 2024-04-01T05:56:03.315 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50054.log.gz 2024-04-01T05:56:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.222876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186696.log 2024-04-01T05:56:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.426183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.476959.log 2024-04-01T05:56:03.317 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.186696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.554246.log 2024-04-01T05:56:03.318 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.476959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.208587.log 2024-04-01T05:56:03.319 INFO:teuthology.orchestra.run.smithi064.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.251067.log.gz 2024-04-01T05:56:03.320 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52350.log.gz 2024-04-01T05:56:03.320 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52266.log.gz/var/log/ceph/ceph-client.admin.554246.log: 2024-04-01T05:56:03.320 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.467751.log.gz 2024-04-01T05:56:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 82.7% -- replaced with /var/log/ceph/ceph-client.admin.227185.log.gz 2024-04-01T05:56:03.321 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.374241.log 2024-04-01T05:56:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.208587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.489977.log 2024-04-01T05:56:03.324 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.374241.log: gzip 78.4% -- replaced with /var/log/ceph/ceph-client.admin.218464.log.gz 2024-04-01T05:56:03.324 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.501674.log 2024-04-01T05:56:03.324 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.422691.log.gz 2024-04-01T05:56:03.324 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.489977.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.220157.log.gz 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2%gzip 90.2% -- replaced with /var/log/ceph/ceph-client.admin.479564.log.gz 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr: -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.180513.log 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.264871.log.gz 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.476959.log.gz 2024-04-01T05:56:03.326 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.501674.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.265713.log.gz 2024-04-01T05:56:03.327 INFO:teuthology.orchestra.run.smithi064.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.469086.log.gz 2024-04-01T05:56:03.328 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48700.log 2024-04-01T05:56:03.328 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.180513.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.554246.log.gz 2024-04-01T05:56:03.328 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.351952.log.gz 2024-04-01T05:56:03.329 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.223358.log 2024-04-01T05:56:03.331 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48700.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.223358.log: /var/log/ceph/ceph-client.admin.44687.log 2024-04-01T05:56:03.331 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44687.log: /var/log/ceph/ceph-client.admin.382739.log 2024-04-01T05:56:03.331 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.382739.log: /var/log/ceph/ceph-client.admin.144491.log 2024-04-01T05:56:03.332 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.502101.log 2024-04-01T05:56:03.333 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.144491.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.462448.log.gz 2024-04-01T05:56:03.335 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.210702.log 2024-04-01T05:56:03.335 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.502101.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.210702.log: /var/log/ceph/ceph-client.admin.236153.log 2024-04-01T05:56:03.336 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140923.log 2024-04-01T05:56:03.337 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.236153.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.140923.log: /var/log/ceph/ceph-client.admin.156054.log 2024-04-01T05:56:03.338 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134572.log 2024-04-01T05:56:03.339 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.156054.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.134572.log: /var/log/ceph/ceph-client.admin.501256.log 2024-04-01T05:56:03.340 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.501256.log: /var/log/ceph/ceph-client.admin.387156.log 2024-04-01T05:56:03.340 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.387156.log: /var/log/ceph/ceph-client.admin.194845.log 2024-04-01T05:56:03.343 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.348136.log.gz 2024-04-01T05:56:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.534321.log 2024-04-01T05:56:03.343 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.180076.log.gz 81.3% -- replaced with /var/log/ceph/ceph-client.admin.348558.log.gz 2024-04-01T05:56:03.343 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.164487.log.gz 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.208587.log.gz 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.208596.log.gz 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.209429.log.gz 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.238727.log 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.194845.log: 81.1% 79.4% -- replaced with /var/log/ceph/ceph-client.admin.382739.log.gz 2024-04-01T05:56:03.344 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.242743.log.gz 2024-04-01T05:56:03.345 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.251508.log.gz 2024-04-01T05:56:03.345 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409607.log 2024-04-01T05:56:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.534321.log: 80.6%/var/log/ceph/ceph-client.admin.238727.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.357665.log.gz 2024-04-01T05:56:03.346 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.266564.log.gz 2024-04-01T05:56:03.346 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.489977.log.gz 2024-04-01T05:56:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.279895.log 2024-04-01T05:56:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133292.log 2024-04-01T05:56:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.409607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.231915.log 2024-04-01T05:56:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.279895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.380533.log 2024-04-01T05:56:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.133292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.409140.log 2024-04-01T05:56:03.349 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.231915.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.260092.log.gz 2024-04-01T05:56:03.349 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.414072.log 2024-04-01T05:56:03.349 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.380533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.541908.log 2024-04-01T05:56:03.350 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.409140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.248660.log 2024-04-01T05:56:03.351 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.414072.log: gzip/var/log/ceph/ceph-client.admin.541908.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.407844.log 2024-04-01T05:56:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.248660.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-04-01T05:56:03.352 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.238727.log.gz 2024-04-01T05:56:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423539.log 2024-04-01T05:56:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.407844.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.541908.log.gz 2024-04-01T05:56:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.503388.log 2024-04-01T05:56:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.423539.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.353222.log.gz 2024-04-01T05:56:03.356 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.508311.log.gz 2024-04-01T05:56:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.406933.log 2024-04-01T05:56:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.503388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.216339.log 2024-04-01T05:56:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.406933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.452965.log 2024-04-01T05:56:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.216339.log: gzip 78.5% -- replaced with /var/log/ceph/ceph-client.admin.194845.log.gz 2024-04-01T05:56:03.361 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.492574.log 2024-04-01T05:56:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.452965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.240035.log 2024-04-01T05:56:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.492574.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.385274.log.gz 2024-04-01T05:56:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.509176.log 2024-04-01T05:56:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.240035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156523.log 2024-04-01T05:56:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.509176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.236144.log 2024-04-01T05:56:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.156523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.516412.log 2024-04-01T05:56:03.369 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.236144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.373811.log 2024-04-01T05:56:03.369 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.516412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.248234.log 2024-04-01T05:56:03.371 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.373811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.531335.log 2024-04-01T05:56:03.372 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.248234.log: gzip -5 --verbose -- 90.2% /var/log/ceph/ceph-client.admin.390767.log 2024-04-01T05:56:03.372 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.385715.log.gz 2024-04-01T05:56:03.374 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.531335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50600.log 2024-04-01T05:56:03.374 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.388495.log.gz 2024-04-01T05:56:03.376 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.390767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.361075.log 2024-04-01T05:56:03.376 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.426183.log.gz 2024-04-01T05:56:03.376 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.499984.log 2024-04-01T05:56:03.376 INFO:teuthology.orchestra.run.smithi064.stderr: 78.9% -- replaced with /var/log/ceph/ceph-client.admin.153046.log.gz 2024-04-01T05:56:03.376 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.361075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160750.log 2024-04-01T05:56:03.377 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.499984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.384020.log 2024-04-01T05:56:03.379 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.160750.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-04-01T05:56:03.379 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.380542.log 2024-04-01T05:56:03.379 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.187578.log.gz 2024-04-01T05:56:03.381 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.384020.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.407418.log.gz 2024-04-01T05:56:03.381 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.548073.log 2024-04-01T05:56:03.381 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46325.log.gz 2024-04-01T05:56:03.383 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50600.log.gz 2024-04-01T05:56:03.383 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188441.log 2024-04-01T05:56:03.383 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.501674.log.gz/var/log/ceph/ceph-client.admin.380542.log: 2024-04-01T05:56:03.384 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50558.log.gz/var/log/ceph/ceph-client.admin.548073.log: 2024-04-01T05:56:03.384 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.501256.log.gz 2024-04-01T05:56:03.384 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50768.log.gz 2024-04-01T05:56:03.384 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124695.log 2024-04-01T05:56:03.384 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.188441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.365290.log 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.124695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.536100.log 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.548073.log.gz 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.493004.log 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.135414.log.gz 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.536100.log: /var/log/ceph/ceph-client.admin.365290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.533030.log 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.536100.log.gz 2024-04-01T05:56:03.388 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.493004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163502.log 2024-04-01T05:56:03.389 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.533030.log: gzip 93.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.378810.log 2024-04-01T05:56:03.389 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.186255.log.gz 2024-04-01T05:56:03.389 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45821.log 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.163502.log: /var/log/ceph/ceph-client.admin.378810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146652.log 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.146652.log: 81.1%/var/log/ceph/ceph-client.admin.45821.log: -- replaced with /var/log/ceph/ceph-client.admin.248660.log.gz 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.506123.log 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121143.log 2024-04-01T05:56:03.394 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.390758.log 2024-04-01T05:56:03.395 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.121143.log: /var/log/ceph/ceph-client.admin.506123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.255403.log 2024-04-01T05:56:03.395 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.428794.log 2024-04-01T05:56:03.395 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.255403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.417062.log 2024-04-01T05:56:03.395 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.428794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.268249.log 2024-04-01T05:56:03.396 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.417062.log: /var/log/ceph/ceph-client.admin.390758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156991.log 2024-04-01T05:56:03.396 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.268249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.493885.log 2024-04-01T05:56:03.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.156991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.260509.log 2024-04-01T05:56:03.397 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37772.log.gz 2024-04-01T05:56:03.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.493885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135405.log 2024-04-01T05:56:03.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.260509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.251930.log 2024-04-01T05:56:03.398 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.135405.log: -- /var/log/ceph/ceph-client.admin.240897.log 2024-04-01T05:56:03.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.251930.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.144491.log.gz 2024-04-01T05:56:03.400 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.388486.log 2024-04-01T05:56:03.400 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.240897.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.112281.log.gz 2024-04-01T05:56:03.400 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.213237.log 2024-04-01T05:56:03.400 INFO:teuthology.orchestra.run.smithi064.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.202895.log.gz 2024-04-01T05:56:03.402 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.388486.log: gzip 80.8% -- replaced with /var/log/ceph/ceph-client.admin.146652.log.gz -5 2024-04-01T05:56:03.402 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.151294.log 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.142753.log.gz 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.143644.log.gz 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.204696.log.gz 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 76.4%/var/log/ceph/ceph-client.admin.213237.log: -- replaced with /var/log/ceph/ceph-client.admin.204246.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114145.log 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2%/var/log/ceph/ceph-client.admin.151294.log: -- replaced with /var/log/ceph/ceph-client.admin.419061.log.gz 2024-04-01T05:56:03.403 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.358086.log 2024-04-01T05:56:03.405 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45541.log 2024-04-01T05:56:03.405 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.358086.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.266976.log.gz 2024-04-01T05:56:03.405 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 84.1% -- replaced with /var/log/ceph/ceph-client.admin.267397.log.gz 2024-04-01T05:56:03.405 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-04-01T05:56:03.406 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.220583.log 2024-04-01T05:56:03.406 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.452965.log.gz 2024-04-01T05:56:03.407 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.51181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.437748.log 2024-04-01T05:56:03.408 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.220583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.552463.log 2024-04-01T05:56:03.409 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.437748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.392794.log 2024-04-01T05:56:03.410 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.552463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.351943.log 2024-04-01T05:56:03.410 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.552463.log.gz 2024-04-01T05:56:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.392794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-04-01T05:56:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.351943.log: gzip -5 78.4% -- replaced with /var/log/ceph/ceph-client.admin.180504.log.gz 2024-04-01T05:56:03.413 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.246484.log 2024-04-01T05:56:03.413 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.188441.log.gz 2024-04-01T05:56:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43776.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.240897.log.gz 2024-04-01T05:56:03.415 INFO:teuthology.orchestra.run.smithi064.stderr: -- 90.2% -- replaced with /var/log/ceph/ceph-client.admin.213237.log.gz 2024-04-01T05:56:03.415 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.364864.log 2024-04-01T05:56:03.415 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.251930.log.gz 2024-04-01T05:56:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.246484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.269512.log 2024-04-01T05:56:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.364864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565121.log 2024-04-01T05:56:03.417 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.269512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.531756.log 2024-04-01T05:56:03.419 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.565121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.227776.log/var/log/ceph/ceph-client.admin.531756.log: 2024-04-01T05:56:03.419 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.565121.log.gz 2024-04-01T05:56:03.419 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157445.log/var/log/ceph/ceph-client.admin.227776.log: 2024-04-01T05:56:03.420 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.269521.log 2024-04-01T05:56:03.422 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.157445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114633.log 2024-04-01T05:56:03.422 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.269521.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.114633.log: /var/log/ceph/ceph-client.admin.51308.log 2024-04-01T05:56:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.499562.log 2024-04-01T05:56:03.424 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.553350.log 2024-04-01T05:56:03.425 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.499562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.281266.log 2024-04-01T05:56:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.553350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.565247.log 2024-04-01T05:56:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 80.8% -- replaced with /var/log/ceph/ceph-client.admin.553350.log.gz 2024-04-01T05:56:03.427 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.281266.log: /var/log/ceph/ceph-client.admin.565247.log: /var/log/ceph/ceph-client.admin.382319.log 2024-04-01T05:56:03.428 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.565247.log.gz 2024-04-01T05:56:03.428 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.414063.log.gz 2024-04-01T05:56:03.428 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.235311.log 2024-04-01T05:56:03.429 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.382319.log: -5/var/log/ceph/ceph-client.admin.235311.log: --verbose -- /var/log/ceph/ceph-client.admin.496959.log 2024-04-01T05:56:03.430 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.198845.log 2024-04-01T05:56:03.431 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.414072.log.gz 2024-04-01T05:56:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.496959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-04-01T05:56:03.432 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.358086.log.gz 2024-04-01T05:56:03.432 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.198845.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-01T05:56:03.432 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.359805.log.gz 2024-04-01T05:56:03.433 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451469.log 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.451469.log: /var/log/ceph/ceph-client.admin.44161.log 81.2% -- replaced with /var/log/ceph/ceph-client.admin.375106.log.gz 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44161.log: /var/log/ceph/ceph-client.admin.488712.log 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% 80.3% -- replaced with /var/log/ceph/ceph-client.admin.531335.log.gz 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.492574.log.gz 2024-04-01T05:56:03.436 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.375591.log.gz 2024-04-01T05:56:03.437 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.374241.log.gz 2024-04-01T05:56:03.437 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.550243.log 2024-04-01T05:56:03.437 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.488712.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.253194.log.gz 2024-04-01T05:56:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.555133.log 2024-04-01T05:56:03.439 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.534321.log.gz 2024-04-01T05:56:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.550243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.211969.log 2024-04-01T05:56:03.440 INFO:teuthology.orchestra.run.smithi064.stderr: 84.5%/var/log/ceph/ceph-client.admin.555133.log: -- replaced with /var/log/ceph/ceph-client.admin.550243.log.gz 2024-04-01T05:56:03.441 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.246930.log.gz 2024-04-01T05:56:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-04-01T05:56:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.211969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120309.log 2024-04-01T05:56:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49010.log: 76.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.537391.log 2024-04-01T05:56:03.444 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.230058.log.gz 2024-04-01T05:56:03.444 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.555133.log.gz 2024-04-01T05:56:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.120309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.225044.log 2024-04-01T05:56:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.537391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355890.log 2024-04-01T05:56:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.225044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47013.log 2024-04-01T05:56:03.447 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.355890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.257098.log 2024-04-01T05:56:03.448 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47013.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.227767.log.gz 2024-04-01T05:56:03.448 INFO:teuthology.orchestra.run.smithi064.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.198845.log.gz 2024-04-01T05:56:03.450 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.236144.log.gz 2024-04-01T05:56:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.485216.log 2024-04-01T05:56:03.450 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.281266.log.gz 2024-04-01T05:56:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.257098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.553802.log 2024-04-01T05:56:03.452 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.126819.log.gz 2024-04-01T05:56:03.452 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.485216.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.192557.log 2024-04-01T05:56:03.452 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.553802.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.359385.log 2024-04-01T05:56:03.452 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.192557.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.499984.log.gz 2024-04-01T05:56:03.453 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.544171.log 2024-04-01T05:56:03.454 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 80.5% -- replaced with /var/log/ceph/ceph-client.admin.388486.log.gz 2024-04-01T05:56:03.455 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1%/var/log/ceph/ceph-client.admin.359385.log: -- replaced with /var/log/ceph/ceph-client.admin.365290.log.gz 2024-04-01T05:56:03.455 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.361075.log.gz 2024-04-01T05:56:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- 90.2% -- replaced with /var/log/ceph/ceph-client.admin.361495.log.gz 2024-04-01T05:56:03.457 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.163072.log 2024-04-01T05:56:03.457 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2024-04-01T05:56:03.457 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.389416.log.gz 2024-04-01T05:56:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.559645.log 2024-04-01T05:56:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.544171.log: 88.2%/var/log/ceph/ceph-client.admin.163072.log: -- replaced with /var/log/ceph/ceph-client.admin.45541.log.gz 2024-04-01T05:56:03.460 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47013.log.gz 2024-04-01T05:56:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-04-01T05:56:03.460 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.475220.log 2024-04-01T05:56:03.460 INFO:teuthology.orchestra.run.smithi064.stderr: 92.9%/var/log/ceph/ceph-client.admin.559645.log: -- replaced with /var/log/ceph/ceph-client.admin.537391.log.gz 2024-04-01T05:56:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181789.log 2024-04-01T05:56:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.475220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127701.log 2024-04-01T05:56:03.463 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.553802.log.gz 90.0% -- replaced with /var/log/ceph/ceph-client.admin.559645.log.gz 2024-04-01T05:56:03.463 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.463 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.181789.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.544171.log.gz 2024-04-01T05:56:03.463 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.378801.log 2024-04-01T05:56:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.127701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.490897.log 2024-04-01T05:56:03.466 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.378801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.556466.log 2024-04-01T05:56:03.466 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.490897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.348567.log 2024-04-01T05:56:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.556466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.434329.log 2024-04-01T05:56:03.469 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.348567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.410450.log 2024-04-01T05:56:03.471 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.434329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.269942.log 2024-04-01T05:56:03.472 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.410450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.270354.log 2024-04-01T05:56:03.473 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.269942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.262223.log 2024-04-01T05:56:03.474 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.270354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.483044.log 2024-04-01T05:56:03.475 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.262223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.491318.log 2024-04-01T05:56:03.476 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.483044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162660.log 2024-04-01T05:56:03.477 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5/var/log/ceph/ceph-client.admin.491318.log: --verbose -- /var/log/ceph/ceph-client.admin.344747.log 2024-04-01T05:56:03.477 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.162660.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.556466.log.gz 2024-04-01T05:56:03.478 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.259244.log 2024-04-01T05:56:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.344747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.214986.log 2024-04-01T05:56:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.436035.log 2024-04-01T05:56:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.259244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.226736.log 2024-04-01T05:56:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.214986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.196182.log 2024-04-01T05:56:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.436035.log: /var/log/ceph/ceph-client.admin.226736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.355479.log 2024-04-01T05:56:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.196182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.451478.log 2024-04-01T05:56:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.355479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.423885.log 2024-04-01T05:56:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.451478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.528373.log/var/log/ceph/ceph-client.admin.423885.log: 2024-04-01T05:56:03.492 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.528373.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.192557.log.gz 2024-04-01T05:56:03.494 INFO:teuthology.orchestra.run.smithi064.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.157445.log.gz 2024-04-01T05:56:03.495 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.257098.log.gz 2024-04-01T05:56:03.496 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.223772.log.gz 2024-04-01T05:56:03.500 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.120730.log.gz 2024-04-01T05:56:03.505 INFO:teuthology.orchestra.run.smithi064.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.485216.log.gz 2024-04-01T05:56:03.505 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.451469.log.gz 2024-04-01T05:56:03.506 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.246484.log.gz 2024-04-01T05:56:03.508 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.344747.log.gz 2024-04-01T05:56:03.510 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% 77.5% -- replaced with /var/log/ceph/ceph-client.admin.163072.log.gz 2024-04-01T05:56:03.510 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.162660.log.gz 2024-04-01T05:56:03.510 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.195304.log.gz 2024-04-01T05:56:03.511 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.181362.log.gz 90.1% -- replaced with /var/log/ceph/ceph-client.admin.375600.log.gz 2024-04-01T05:56:03.512 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:03.512 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.181789.log.gz 2024-04-01T05:56:03.513 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.377536.log.gz 2024-04-01T05:56:03.514 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.379222.log.gz 2024-04-01T05:56:03.514 INFO:teuthology.orchestra.run.smithi064.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.378801.log.gz 2024-04-01T05:56:03.515 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.359385.log.gz 2024-04-01T05:56:03.515 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.270354.log.gz 2024-04-01T05:56:03.515 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.272113.log.gz 2024-04-01T05:56:03.517 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.382319.log.gz 2024-04-01T05:56:03.517 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.266985.log.gz 2024-04-01T05:56:03.517 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.269942.log.gz 2024-04-01T05:56:03.525 INFO:teuthology.orchestra.run.smithi064.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.114145.log.gz 2024-04-01T05:56:03.526 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.215886.log.gz 2024-04-01T05:56:03.527 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-04-01T05:56:03.527 INFO:teuthology.orchestra.run.smithi064.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.488712.log.gz 2024-04-01T05:56:03.543 INFO:teuthology.orchestra.run.smithi064.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.561812.log.gz 2024-04-01T05:56:03.545 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-04-01T05:56:03.558 INFO:teuthology.orchestra.run.smithi064.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.235302.log.gz 2024-04-01T05:56:03.558 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.253703.log.gz 2024-04-01T05:56:03.559 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-04-01T05:56:03.559 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.283386.log.gz 2024-04-01T05:56:03.561 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.279406.log.gz 2024-04-01T05:56:03.562 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.365712.log.gz 2024-04-01T05:56:03.562 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.406496.log.gz 2024-04-01T05:56:03.563 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46157.log.gz 2024-04-01T05:56:03.564 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.367044.log.gz 2024-04-01T05:56:03.565 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.239150.log.gz 2024-04-01T05:56:03.568 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48742.log.gz 2024-04-01T05:56:03.571 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.119404.log.gz 2024-04-01T05:56:03.573 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48251.log.gz 2024-04-01T05:56:03.574 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51223.log.gz 2024-04-01T05:56:03.577 INFO:teuthology.orchestra.run.smithi064.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49508.log.gz 2024-04-01T05:56:03.579 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50306.log.gz 2024-04-01T05:56:03.579 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.140914.log.gz 2024-04-01T05:56:03.589 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.475220.log.gz 2024-04-01T05:56:03.594 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44435.log.gz 2024-04-01T05:56:03.610 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.131561.log.gz 2024-04-01T05:56:03.611 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.152195.log.gz 2024-04-01T05:56:03.612 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.423530.log.gz 2024-04-01T05:56:03.614 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.112272.log.gz 2024-04-01T05:56:03.615 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.418640.log.gz 2024-04-01T05:56:03.615 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.257966.log.gz 2024-04-01T05:56:03.615 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.263608.log.gz 2024-04-01T05:56:03.615 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.224614.log.gz 2024-04-01T05:56:03.616 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.233179.log.gz 2024-04-01T05:56:03.616 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.349825.log.gz 2024-04-01T05:56:03.616 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.347710.log.gz 2024-04-01T05:56:03.617 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.207745.log.gz 2024-04-01T05:56:03.618 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.466904.log.gz 2024-04-01T05:56:03.619 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.119879.log.gz 2024-04-01T05:56:03.619 INFO:teuthology.orchestra.run.smithi064.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.381881.log.gz 2024-04-01T05:56:03.620 INFO:teuthology.orchestra.run.smithi064.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.187569.log.gz 2024-04-01T05:56:03.620 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.269933.log.gz 2024-04-01T05:56:03.620 INFO:teuthology.orchestra.run.smithi064.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.413618.log.gz 2024-04-01T05:56:03.620 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.278512.log.gz 2024-04-01T05:56:03.621 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.415363.log.gz 2024-04-01T05:56:03.646 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.189357.log.gz 2024-04-01T05:56:03.649 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.229138.log.gz 2024-04-01T05:56:03.668 INFO:teuthology.orchestra.run.smithi064.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.197973.log.gz 2024-04-01T05:56:03.669 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.352372.log.gz 2024-04-01T05:56:03.670 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.238314.log.gz 2024-04-01T05:56:03.672 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.364430.log.gz 2024-04-01T05:56:03.673 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.407835.log.gz 2024-04-01T05:56:03.673 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.368301.log.gz 2024-04-01T05:56:03.673 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.369995.log.gz 2024-04-01T05:56:03.678 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45540.log.gz 88.9% 2024-04-01T05:56:03.678 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45204.log.gz 2024-04-01T05:56:03.679 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46283.log.gz 2024-04-01T05:56:03.680 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48293.log.gz 2024-04-01T05:56:03.680 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-04-01T05:56:03.681 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2024-04-01T05:56:03.682 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 88.2% 2024-04-01T05:56:03.683 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49052.log.gz 2024-04-01T05:56:03.683 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49582.log.gz 2024-04-01T05:56:03.683 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44245.log.gz 2024-04-01T05:56:03.693 INFO:teuthology.orchestra.run.smithi064.stderr: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.230599.log.gz 2024-04-01T05:56:03.697 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.355890.log.gz 2024-04-01T05:56:03.698 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.355479.log.gz 2024-04-01T05:56:03.707 INFO:teuthology.orchestra.run.smithi064.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.185370.log.gz 2024-04-01T05:56:03.707 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.224202.log.gz 2024-04-01T05:56:03.707 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.223781.log.gz 2024-04-01T05:56:03.714 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.493004.log.gz 2024-04-01T05:56:03.715 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.196182.log.gz 2024-04-01T05:56:03.718 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.137532.log.gz 2024-04-01T05:56:03.719 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.218885.log.gz 2024-04-01T05:56:03.721 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.453806.log.gz 2024-04-01T05:56:03.721 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.221613.log.gz 2024-04-01T05:56:03.721 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.465000.log.gz 2024-04-01T05:56:03.722 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.445616.log.gz 2024-04-01T05:56:03.723 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.161257.log.gz 2024-04-01T05:56:03.723 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.232336.log.gz 2024-04-01T05:56:03.723 INFO:teuthology.orchestra.run.smithi064.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.181780.log.gz 2024-04-01T05:56:03.724 INFO:teuthology.orchestra.run.smithi064.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.207324.log.gz 2024-04-01T05:56:03.724 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.344334.log.gz 2024-04-01T05:56:03.725 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.208166.log.gz 2024-04-01T05:56:03.725 INFO:teuthology.orchestra.run.smithi064.stderr: 79.6% -- replaced with /var/log/ceph/ceph-client.admin.377051.log.gz 2024-04-01T05:56:03.726 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.189348.log.gz 2024-04-01T05:56:03.727 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.249090.log.gz 2024-04-01T05:56:03.727 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.354176.log.gz 2024-04-01T05:56:03.728 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.228677.log.gz 2024-04-01T05:56:03.729 INFO:teuthology.orchestra.run.smithi064.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.268240.log.gz 2024-04-01T05:56:03.740 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.489133.log.gz 2024-04-01T05:56:03.758 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.201104.log.gz 2024-04-01T05:56:03.760 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.343065.log.gz 2024-04-01T05:56:03.761 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-04-01T05:56:03.761 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-04-01T05:56:03.794 INFO:teuthology.orchestra.run.smithi064.stderr: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.225035.log.gz 2024-04-01T05:56:03.798 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.142744.log.gz 2024-04-01T05:56:03.798 INFO:teuthology.orchestra.run.smithi064.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.144909.log.gz 2024-04-01T05:56:03.802 INFO:teuthology.orchestra.run.smithi064.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.216330.log.gz 2024-04-01T05:56:03.804 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.260509.log.gz 2024-04-01T05:56:03.808 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.261371.log.gz 2024-04-01T05:56:03.808 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.261797.log.gz 2024-04-01T05:56:03.816 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.129412.log.gz 2024-04-01T05:56:03.827 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.270363.log.gz 2024-04-01T05:56:03.827 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.197082.log.gz 2024-04-01T05:56:03.830 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.416203.log.gz 2024-04-01T05:56:03.831 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.357253.log.gz 2024-04-01T05:56:03.834 INFO:teuthology.orchestra.run.smithi064.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.147075.log.gz 2024-04-01T05:56:03.844 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.211960.log.gz 2024-04-01T05:56:03.847 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.242322.log.gz 2024-04-01T05:56:03.858 INFO:teuthology.orchestra.run.smithi064.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.488098.log.gz 2024-04-01T05:56:03.859 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.491309.log.gz 2024-04-01T05:56:03.863 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.385283.log.gz 2024-04-01T05:56:03.865 INFO:teuthology.orchestra.run.smithi064.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.273431.log.gz 2024-04-01T05:56:03.866 INFO:teuthology.orchestra.run.smithi064.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.343056.log.gz 2024-04-01T05:56:03.871 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.496100.log.gz 2024-04-01T05:56:03.874 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.357674.log.gz 2024-04-01T05:56:03.879 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.441145.log.gz 2024-04-01T05:56:03.879 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.442020.log.gz 2024-04-01T05:56:03.879 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.352363.log.gz 2024-04-01T05:56:03.882 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.436454.log.gz 2024-04-01T05:56:03.883 INFO:teuthology.orchestra.run.smithi064.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.371276.log.gz 2024-04-01T05:56:03.883 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.369153.log.gz 2024-04-01T05:56:03.884 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.364873.log.gz 2024-04-01T05:56:03.884 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.366197.log.gz 2024-04-01T05:56:03.885 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.425766.log.gz 2024-04-01T05:56:03.892 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.134142.log.gz 2024-04-01T05:56:03.896 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.365721.log.gz 2024-04-01T05:56:03.896 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% 81.1% -- replaced with /var/log/ceph/ceph-client.admin.486188.log.gz 2024-04-01T05:56:03.896 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.485703.log.gz 2024-04-01T05:56:03.897 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.462027.log.gz 2024-04-01T05:56:03.900 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.432217.log.gz 2024-04-01T05:56:03.917 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.143635.log.gz 2024-04-01T05:56:03.918 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.222867.log.gz 2024-04-01T05:56:03.921 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.480050.log.gz 2024-04-01T05:56:03.927 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.220592.log.gz 2024-04-01T05:56:03.927 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.195728.log.gz 2024-04-01T05:56:03.930 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.262635.log.gz 2024-04-01T05:56:03.930 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.268249.log.gz 2024-04-01T05:56:03.932 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.264880.log.gz 2024-04-01T05:56:03.933 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.476117.log.gz 2024-04-01T05:56:03.936 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.448869.log.gz 2024-04-01T05:56:03.936 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.481528.log.gz 2024-04-01T05:56:03.937 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.453385.log.gz 2024-04-01T05:56:03.940 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.125161.log.gz 2024-04-01T05:56:03.941 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.442441.log.gz 2024-04-01T05:56:03.946 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.378378.log.gz 2024-04-01T05:56:03.946 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.186246.log.gz 2024-04-01T05:56:03.947 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.278950.log.gz 2024-04-01T05:56:03.947 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.416634.log.gz 2024-04-01T05:56:03.951 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.496950.log.gz 2024-04-01T05:56:03.951 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.497372.log.gz 2024-04-01T05:56:03.953 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-01T05:56:03.953 INFO:teuthology.orchestra.run.smithi064.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.151285.log.gz 2024-04-01T05:56:03.973 INFO:teuthology.orchestra.run.smithi064.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.529638.log.gz 2024-04-01T05:56:03.974 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.491741.log.gz 2024-04-01T05:56:03.984 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.484339.log.gz 2024-04-01T05:56:03.988 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.347719.log.gz 2024-04-01T05:56:03.989 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.412356.log.gz 2024-04-01T05:56:04.003 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.439455.log.gz 2024-04-01T05:56:04.005 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.386671.log.gz 2024-04-01T05:56:04.008 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48083.log.gz 2024-04-01T05:56:04.009 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50180.log.gz 2024-04-01T05:56:04.009 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-04-01T05:56:04.010 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2024-04-01T05:56:04.022 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.509597.log.gz 2024-04-01T05:56:04.027 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.205156.log.gz 2024-04-01T05:56:04.029 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.161266.log.gz 2024-04-01T05:56:04.029 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2024-04-01T05:56:04.031 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.203354.log.gz 2024-04-01T05:56:04.035 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.129843.log.gz 2024-04-01T05:56:04.038 INFO:teuthology.orchestra.run.smithi064.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.115064.log.gz 2024-04-01T05:56:04.046 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.475229.log.gz 2024-04-01T05:56:04.047 INFO:teuthology.orchestra.run.smithi064.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.481041.log.gz 2024-04-01T05:56:04.051 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.249503.log.gz 2024-04-01T05:56:04.053 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.254986.log.gz 2024-04-01T05:56:04.054 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.256681.log.gz 2024-04-01T05:56:04.061 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.479141.log.gz 2024-04-01T05:56:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.413197.log.gz 2024-04-01T05:56:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.416625.log.gz 2024-04-01T05:56:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.416212.log.gz 2024-04-01T05:56:04.077 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.413627.log.gz 2024-04-01T05:56:04.094 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.505674.log.gz 2024-04-01T05:56:04.096 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.488721.log.gz 2024-04-01T05:56:04.097 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.431366.log.gz 2024-04-01T05:56:04.098 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.492153.log.gz 2024-04-01T05:56:04.103 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.442862.log.gz 2024-04-01T05:56:04.104 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.241340.log.gz 2024-04-01T05:56:04.105 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.272535.log.gz 2024-04-01T05:56:04.108 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.196173.log.gz 2024-04-01T05:56:04.109 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.421810.log.gz 2024-04-01T05:56:04.118 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.373820.log.gz 2024-04-01T05:56:04.120 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.522970.log.gz 2024-04-01T05:56:04.124 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.182637.log.gz 2024-04-01T05:56:04.135 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.369574.log.gz 2024-04-01T05:56:04.142 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.132454.log.gz 2024-04-01T05:56:04.145 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46031.log.gz 2024-04-01T05:56:04.154 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.533872.log.gz 2024-04-01T05:56:04.156 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.138791.log.gz 2024-04-01T05:56:04.159 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.140497.log.gz 2024-04-01T05:56:04.182 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.254125.log.gz 2024-04-01T05:56:04.183 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.254134.log.gz 2024-04-01T05:56:04.184 INFO:teuthology.orchestra.run.smithi064.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.267827.log.gz 2024-04-01T05:56:04.187 INFO:teuthology.orchestra.run.smithi064.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.260945.log.gz 2024-04-01T05:56:04.190 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.525419.log.gz 2024-04-01T05:56:04.192 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.525428.log.gz 2024-04-01T05:56:04.206 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.452539.log.gz 2024-04-01T05:56:04.207 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.154364.log.gz 2024-04-01T05:56:04.213 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.498231.log.gz 2024-04-01T05:56:04.214 INFO:teuthology.orchestra.run.smithi064.stderr: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.506603.log.gz 2024-04-01T05:56:04.215 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.185804.log.gz 2024-04-01T05:56:04.217 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.508751.log.gz 2024-04-01T05:56:04.218 INFO:teuthology.orchestra.run.smithi064.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.141382.log.gz 2024-04-01T05:56:04.218 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.510048.log.gz 2024-04-01T05:56:04.220 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.348145.log.gz 2024-04-01T05:56:04.220 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51732.log.gz 2024-04-01T05:56:04.221 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51960.log.gz 2024-04-01T05:56:04.222 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.419590.log.gz 2024-04-01T05:56:04.222 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.420911.log.gz 2024-04-01T05:56:04.223 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.495670.log.gz 2024-04-01T05:56:04.230 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52416.log.gz 2024-04-01T05:56:04.230 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.491732.log.gz 2024-04-01T05:56:04.244 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.391775.log.gz 2024-04-01T05:56:04.265 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.436463.log.gz 2024-04-01T05:56:04.267 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.483919.log.gz 2024-04-01T05:56:04.268 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.483493.log.gz 2024-04-01T05:56:04.273 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.438612.log.gz 2024-04-01T05:56:04.279 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-04-01T05:56:04.280 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-04-01T05:56:04.280 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50222.log.gz 2024-04-01T05:56:04.281 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51265.log.gz 2024-04-01T05:56:04.293 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.428329.log.gz 2024-04-01T05:56:04.296 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.204255.log.gz 2024-04-01T05:56:04.297 INFO:teuthology.orchestra.run.smithi064.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.501247.log.gz 2024-04-01T05:56:04.297 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.498717.log.gz 2024-04-01T05:56:04.315 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52224.log.gz 2024-04-01T05:56:04.315 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.219727.log.gz 2024-04-01T05:56:04.317 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.197532.log.gz 2024-04-01T05:56:04.324 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.469508.log.gz 2024-04-01T05:56:04.331 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.475642.log.gz 2024-04-01T05:56:04.331 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.480554.log.gz 2024-04-01T05:56:04.331 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.259666.log.gz 2024-04-01T05:56:04.336 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.469517.log.gz 2024-04-01T05:56:04.353 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.451012.log.gz 2024-04-01T05:56:04.355 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.213228.log.gz 2024-04-01T05:56:04.355 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.232749.log.gz 2024-04-01T05:56:04.355 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.212815.log.gz 2024-04-01T05:56:04.356 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.267818.log.gz 2024-04-01T05:56:04.357 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.377060.log.gz 2024-04-01T05:56:04.357 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.473454.log.gz 2024-04-01T05:56:04.365 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.120300.log.gz 2024-04-01T05:56:04.367 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.128579.log.gz 2024-04-01T05:56:04.370 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.470781.log.gz 2024-04-01T05:56:04.372 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.471211.log.gz 2024-04-01T05:56:04.372 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.472472.log.gz 2024-04-01T05:56:04.374 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.530922.log.gz 2024-04-01T05:56:04.375 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.492583.log.gz 2024-04-01T05:56:04.377 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.438188.log.gz 2024-04-01T05:56:04.381 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.447602.log.gz 2024-04-01T05:56:04.383 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48926.log.gz 2024-04-01T05:56:04.384 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.462870.log.gz 2024-04-01T05:56:04.385 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz 2024-04-01T05:56:04.400 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.249081.log.gz 2024-04-01T05:56:04.401 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.358550.log.gz 2024-04-01T05:56:04.401 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.113663.log.gz 2024-04-01T05:56:04.401 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52308.log.gz 2024-04-01T05:56:04.410 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-04-01T05:56:04.415 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2024-04-01T05:56:04.421 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.232327.log.gz 2024-04-01T05:56:04.442 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.377957.log.gz 2024-04-01T05:56:04.473 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.437757.log.gz 2024-04-01T05:56:04.483 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.233624.log.gz 2024-04-01T05:56:04.494 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.282961.log.gz 2024-04-01T05:56:04.496 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43953.log.gz 2024-04-01T05:56:04.499 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.239572.log.gz 2024-04-01T05:56:04.500 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.507459.log.gz 2024-04-01T05:56:04.501 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.435609.log.gz 2024-04-01T05:56:04.503 INFO:teuthology.orchestra.run.smithi064.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.199745.log.gz 2024-04-01T05:56:04.505 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.370851.log.gz 2024-04-01T05:56:04.506 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.234890.log.gz 2024-04-01T05:56:04.508 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.368310.log.gz 2024-04-01T05:56:04.509 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.428785.log.gz 2024-04-01T05:56:04.515 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-04-01T05:56:04.515 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50432.log.gz 2024-04-01T05:56:04.517 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2024-04-01T05:56:04.521 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.543937.log.gz 2024-04-01T05:56:04.524 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.127251.log.gz 2024-04-01T05:56:04.525 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.194854.log.gz 2024-04-01T05:56:04.526 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.127710.log.gz 2024-04-01T05:56:04.530 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.130684.log.gz 2024-04-01T05:56:04.534 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.147954.log.gz 2024-04-01T05:56:04.537 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.112721.log.gz 2024-04-01T05:56:04.539 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.217624.log.gz 2024-04-01T05:56:04.541 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.470790.log.gz 2024-04-01T05:56:04.542 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.477380.log.gz 2024-04-01T05:56:04.546 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.362335.log.gz 2024-04-01T05:56:04.546 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.255838.log.gz 2024-04-01T05:56:04.547 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.258818.log.gz 2024-04-01T05:56:04.549 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.265301.log.gz 2024-04-01T05:56:04.550 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.205607.log.gz 2024-04-01T05:56:04.564 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47747.log.gz 2024-04-01T05:56:04.581 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.226727.log.gz 2024-04-01T05:56:04.582 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.358541.log.gz 2024-04-01T05:56:04.583 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.153480.log.gz 2024-04-01T05:56:04.586 INFO:teuthology.orchestra.run.smithi064.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.161710.log.gz 2024-04-01T05:56:04.588 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.163081.log.gz 2024-04-01T05:56:04.589 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.451877.log.gz 2024-04-01T05:56:04.591 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.358974.log.gz 2024-04-01T05:56:04.592 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.529647.log.gz 2024-04-01T05:56:04.599 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.246493.log.gz 2024-04-01T05:56:04.602 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.197982.log.gz 2024-04-01T05:56:04.603 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-04-01T05:56:04.604 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.509167.log.gz 2024-04-01T05:56:04.608 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.423789.log.gz 2024-04-01T05:56:04.609 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.390273.log.gz 2024-04-01T05:56:04.613 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.386191.log.gz 2024-04-01T05:56:04.613 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.388908.log.gz 2024-04-01T05:56:04.615 INFO:teuthology.orchestra.run.smithi064.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.132445.log.gz 2024-04-01T05:56:04.617 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.180940.log.gz 2024-04-01T05:56:04.630 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-04-01T05:56:04.630 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.358095.log.gz 2024-04-01T05:56:04.645 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.409616.log.gz 2024-04-01T05:56:04.646 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.385706.log.gz 2024-04-01T05:56:04.665 INFO:teuthology.orchestra.run.smithi064.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.193399.log.gz 2024-04-01T05:56:04.672 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.220148.log.gz 2024-04-01T05:56:04.673 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.187128.log.gz 2024-04-01T05:56:04.680 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.262214.log.gz 2024-04-01T05:56:04.683 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.513750.log.gz 2024-04-01T05:56:04.696 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.118557.log.gz 2024-04-01T05:56:04.710 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.535186.log.gz 2024-04-01T05:56:04.727 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.211537.log.gz 2024-04-01T05:56:04.729 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.359394.log.gz 2024-04-01T05:56:04.729 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.272104.log.gz 2024-04-01T05:56:04.730 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.179652.log.gz 2024-04-01T05:56:04.731 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.420011.log.gz 2024-04-01T05:56:04.732 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.187119.log.gz 2024-04-01T05:56:04.734 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.186705.log.gz 2024-04-01T05:56:04.744 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.362326.log.gz 2024-04-01T05:56:04.749 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.368730.log.gz 2024-04-01T05:56:04.751 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48041.log.gz 2024-04-01T05:56:04.755 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-04-01T05:56:04.756 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50726.log.gz 2024-04-01T05:56:04.772 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.504686.log.gz 2024-04-01T05:56:04.795 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.215436.log.gz 2024-04-01T05:56:04.795 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.202904.log.gz 2024-04-01T05:56:04.795 INFO:teuthology.orchestra.run.smithi064.stderr: 82.6% 94.8% -- replaced with /var/log/ceph/ceph-client.admin.160315.log.gz -- replaced with /var/log/ceph/ceph-client.0.52676.log.gz 2024-04-01T05:56:04.796 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:04.802 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.134993.log.gz 2024-04-01T05:56:04.803 INFO:teuthology.orchestra.run.smithi064.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.137523.log.gz 2024-04-01T05:56:04.805 INFO:teuthology.orchestra.run.smithi064.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.127260.log.gz 2024-04-01T05:56:04.806 INFO:teuthology.orchestra.run.smithi064.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.112712.log.gz 2024-04-01T05:56:04.825 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.182206.log.gz 2024-04-01T05:56:04.826 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.349834.log.gz 2024-04-01T05:56:04.832 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.379677.log.gz 2024-04-01T05:56:04.834 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.415791.log.gz 2024-04-01T05:56:04.835 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.360646.log.gz 2024-04-01T05:56:04.838 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.367035.log.gz 2024-04-01T05:56:04.854 INFO:teuthology.orchestra.run.smithi064.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.443283.log.gz 2024-04-01T05:56:04.859 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.376076.log.gz 2024-04-01T05:56:04.867 INFO:teuthology.orchestra.run.smithi064.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.506378.log.gz 2024-04-01T05:56:04.871 INFO:teuthology.orchestra.run.smithi064.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.234459.log.gz 2024-04-01T05:56:04.880 INFO:teuthology.orchestra.run.smithi064.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.235723.log.gz 2024-04-01T05:56:04.882 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.236626.log.gz 2024-04-01T05:56:04.882 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.237470.log.gz 2024-04-01T05:56:04.883 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.237892.log.gz 2024-04-01T05:56:04.883 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.429649.log.gz 2024-04-01T05:56:04.885 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.387147.log.gz 2024-04-01T05:56:04.885 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.430949.log.gz 2024-04-01T05:56:04.886 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.134151.log.gz 2024-04-01T05:56:04.887 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.384440.log.gz 2024-04-01T05:56:04.887 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.462018.log.gz 2024-04-01T05:56:04.889 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.512504.log.gz 2024-04-01T05:56:04.892 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-04-01T05:56:04.892 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.161719.log.gz 2024-04-01T05:56:04.892 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-04-01T05:56:04.894 INFO:teuthology.orchestra.run.smithi064.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.351523.log.gz 2024-04-01T05:56:04.894 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48437.log.gz 2024-04-01T05:56:04.896 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.408270.log.gz 2024-04-01T05:56:04.897 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47663.log.gz 2024-04-01T05:56:04.904 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.134984.log.gz 2024-04-01T05:56:04.904 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.136256.log.gz 2024-04-01T05:56:04.905 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-04-01T05:56:04.910 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.204705.log.gz 2024-04-01T05:56:04.924 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.503397.log.gz 2024-04-01T05:56:04.924 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.222455.log.gz 2024-04-01T05:56:04.940 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.261362.log.gz 2024-04-01T05:56:04.944 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.482015.log.gz 2024-04-01T05:56:04.956 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.448452.log.gz 2024-04-01T05:56:04.986 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.467316.log.gz 2024-04-01T05:56:04.994 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.379686.log.gz 2024-04-01T05:56:04.995 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.163493.log.gz 2024-04-01T05:56:04.998 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.243586.log.gz 2024-04-01T05:56:04.999 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.414498.log.gz 2024-04-01T05:56:05.000 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.242752.log.gz 2024-04-01T05:56:05.002 INFO:teuthology.orchestra.run.smithi064.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.346442.log.gz 2024-04-01T05:56:05.002 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.343912.log.gz 2024-04-01T05:56:05.003 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.346863.log.gz 2024-04-01T05:56:05.009 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.493464.log.gz 2024-04-01T05:56:05.012 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.268672.log.gz 2024-04-01T05:56:05.013 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.163808.log.gz 2024-04-01T05:56:05.014 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.354637.log.gz 2024-04-01T05:56:05.016 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.188895.log.gz 2024-04-01T05:56:05.018 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.189854.log.gz 2024-04-01T05:56:05.032 INFO:teuthology.orchestra.run.smithi064.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.474788.log.gz 2024-04-01T05:56:05.033 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.478707.log.gz 2024-04-01T05:56:05.051 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.387575.log.gz 2024-04-01T05:56:05.053 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.534745.log.gz 2024-04-01T05:56:05.074 INFO:teuthology.orchestra.run.smithi064.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.128991.log.gz 2024-04-01T05:56:05.075 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50474.log.gz 2024-04-01T05:56:05.081 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.155627.log.gz 2024-04-01T05:56:05.081 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.154793.log.gz 2024-04-01T05:56:05.082 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.136680.log.gz 2024-04-01T05:56:05.085 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.228686.log.gz 2024-04-01T05:56:05.087 INFO:teuthology.orchestra.run.smithi064.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.461549.log.gz 2024-04-01T05:56:05.089 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.502943.log.gz 2024-04-01T05:56:05.091 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.254977.log.gz 2024-04-01T05:56:05.091 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.502530.log.gz 2024-04-01T05:56:05.093 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.350677.log.gz 2024-04-01T05:56:05.094 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.350686.log.gz 2024-04-01T05:56:05.096 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.222034.log.gz 2024-04-01T05:56:05.096 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.363176.log.gz 2024-04-01T05:56:05.097 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.364018.log.gz 2024-04-01T05:56:05.111 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.156054.log.gz 2024-04-01T05:56:05.113 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.198404.log.gz 2024-04-01T05:56:05.113 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.345603.log.gz 2024-04-01T05:56:05.122 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.223358.log.gz 2024-04-01T05:56:05.122 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.225885.log.gz 2024-04-01T05:56:05.124 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.247385.log.gz 2024-04-01T05:56:05.124 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.376566.log.gz 2024-04-01T05:56:05.125 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.186696.log.gz 2024-04-01T05:56:05.126 INFO:teuthology.orchestra.run.smithi064.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.414489.log.gz 2024-04-01T05:56:05.127 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.412786.log.gz 2024-04-01T05:56:05.131 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.260083.log.gz 2024-04-01T05:56:05.154 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.382310.log.gz 2024-04-01T05:56:05.156 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.234046.log.gz 2024-04-01T05:56:05.158 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.180513.log.gz 2024-04-01T05:56:05.159 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.351943.log.gz 2024-04-01T05:56:05.160 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.375115.log.gz 2024-04-01T05:56:05.165 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.451021.log.gz 2024-04-01T05:56:05.168 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.132023.log.gz 2024-04-01T05:56:05.170 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.360235.log.gz 2024-04-01T05:56:05.181 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.153926.log.gz 2024-04-01T05:56:05.181 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45821.log.gz 2024-04-01T05:56:05.183 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.115495.log.gz 2024-04-01T05:56:05.184 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2024-04-01T05:56:05.192 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44687.log.gz 2024-04-01T05:56:05.206 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.473866.log.gz 2024-04-01T05:56:05.207 INFO:teuthology.orchestra.run.smithi064.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.129834.log.gz 2024-04-01T05:56:05.213 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.224623.log.gz 2024-04-01T05:56:05.217 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.133292.log.gz 90.1% 2024-04-01T05:56:05.217 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.431796.log.gz 2024-04-01T05:56:05.217 INFO:teuthology.orchestra.run.smithi064.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.131109.log.gz 2024-04-01T05:56:05.219 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.156991.log.gz 2024-04-01T05:56:05.219 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.220583.log.gz 2024-04-01T05:56:05.225 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.473017.log.gz -- replaced with /var/log/ceph/ceph-client.admin.471633.log.gz 2024-04-01T05:56:05.225 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:05.243 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.201545.log.gz 2024-04-01T05:56:05.276 INFO:teuthology.orchestra.run.smithi064.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.373811.log.gz 2024-04-01T05:56:05.277 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.380533.log.gz 2024-04-01T05:56:05.277 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.188019.log.gz 2024-04-01T05:56:05.278 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.376575.log.gz 2024-04-01T05:56:05.295 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.503388.log.gz 2024-04-01T05:56:05.311 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.231915.log.gz 2024-04-01T05:56:05.317 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.384020.log.gz 2024-04-01T05:56:05.317 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.132866.log.gz 2024-04-01T05:56:05.319 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.387156.log.gz 2024-04-01T05:56:05.324 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.525007.log.gz 2024-04-01T05:56:05.329 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48521.log.gz 2024-04-01T05:56:05.331 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.0.46890.log.gz 2024-04-01T05:56:05.342 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2024-04-01T05:56:05.343 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51308.log.gz 2024-04-01T05:56:05.349 INFO:teuthology.orchestra.run.smithi064.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.144482.log.gz 2024-04-01T05:56:05.349 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.224193.log.gz 2024-04-01T05:56:05.363 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.218894.log.gz 2024-04-01T05:56:05.363 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.145351.log.gz 2024-04-01T05:56:05.368 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.144918.log.gz 2024-04-01T05:56:05.407 INFO:teuthology.orchestra.run.smithi064.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.249979.log.gz 2024-04-01T05:56:05.409 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.493885.log.gz 2024-04-01T05:56:05.411 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.378810.log.gz 2024-04-01T05:56:05.427 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.198854.log.gz 2024-04-01T05:56:05.466 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.254551.log.gz 2024-04-01T05:56:05.467 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.282115.log.gz 2024-04-01T05:56:05.468 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.429640.log.gz 2024-04-01T05:56:05.470 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.360655.log.gz 2024-04-01T05:56:05.476 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.364864.log.gz 2024-04-01T05:56:05.476 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.367457.log.gz 2024-04-01T05:56:05.477 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.437748.log.gz 2024-04-01T05:56:05.478 INFO:teuthology.orchestra.run.smithi064.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.137101.log.gz 2024-04-01T05:56:05.478 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-04-01T05:56:05.479 INFO:teuthology.orchestra.run.smithi064.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.2.47389.log.gz 2024-04-01T05:56:05.486 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.225044.log.gz 2024-04-01T05:56:05.488 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.183066.log.gz 2024-04-01T05:56:05.494 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.114633.log.gz 2024-04-01T05:56:05.495 INFO:teuthology.orchestra.run.smithi064.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.124686.log.gz 2024-04-01T05:56:05.495 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.153055.log.gz 2024-04-01T05:56:05.496 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.206062.log.gz 2024-04-01T05:56:05.499 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.248234.log.gz 2024-04-01T05:56:05.502 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.255412.log.gz 2024-04-01T05:56:05.507 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.428794.log.gz 2024-04-01T05:56:05.508 INFO:teuthology.orchestra.run.smithi064.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.130675.log.gz 2024-04-01T05:56:05.510 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.130263.log.gz 2024-04-01T05:56:05.525 INFO:teuthology.orchestra.run.smithi064.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.355470.log.gz 2024-04-01T05:56:05.536 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.417053.log.gz 2024-04-01T05:56:05.549 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.412365.log.gz 2024-04-01T05:56:05.551 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.483044.log.gz 2024-04-01T05:56:05.552 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.210702.log.gz 2024-04-01T05:56:05.553 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.211123.log.gz 2024-04-01T05:56:05.583 INFO:teuthology.orchestra.run.smithi064.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.195295.log.gz 2024-04-01T05:56:05.583 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.237461.log.gz 2024-04-01T05:56:05.585 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.254560.log.gz 2024-04-01T05:56:05.588 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.496959.log.gz 2024-04-01T05:56:05.592 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49298.log.gz 2024-04-01T05:56:05.601 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.243595.log.gz 2024-04-01T05:56:05.601 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.282124.log.gz 2024-04-01T05:56:05.611 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44287.log.gz 2024-04-01T05:56:05.613 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.183516.log.gz 2024-04-01T05:56:05.617 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46367.log.gz 2024-04-01T05:56:05.617 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.137944.log.gz 2024-04-01T05:56:05.619 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.468641.log.gz 2024-04-01T05:56:05.620 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.140071.log.gz 2024-04-01T05:56:05.627 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.225456.log.gz 2024-04-01T05:56:05.631 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.194273.log.gz 2024-04-01T05:56:05.632 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.119888.log.gz 2024-04-01T05:56:05.634 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-04-01T05:56:05.635 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45200.log.gz 2024-04-01T05:56:05.644 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.410450.log.gz 2024-04-01T05:56:05.651 INFO:teuthology.orchestra.run.smithi064.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.151764.log.gz 2024-04-01T05:56:05.659 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.392288.log.gz 2024-04-01T05:56:05.665 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.264029.log.gz 2024-04-01T05:56:05.681 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.348993.log.gz 2024-04-01T05:56:05.681 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.348567.log.gz 2024-04-01T05:56:05.682 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.358965.log.gz 2024-04-01T05:56:05.684 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.210271.log.gz 2024-04-01T05:56:05.686 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.162651.log.gz 2024-04-01T05:56:05.705 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.490897.log.gz 2024-04-01T05:56:05.707 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.492162.log.gz 2024-04-01T05:56:05.710 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.196623.log.gz 2024-04-01T05:56:05.711 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.234037.log.gz 2024-04-01T05:56:05.719 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.366613.log.gz 2024-04-01T05:56:05.721 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2024-04-01T05:56:05.721 INFO:teuthology.orchestra.run.smithi064.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.423885.log.gz 2024-04-01T05:56:05.731 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.118313.log.gz 2024-04-01T05:56:05.732 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.202004.log.gz 2024-04-01T05:56:05.735 INFO:teuthology.orchestra.run.smithi064.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.221183.log.gz 2024-04-01T05:56:05.741 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.463282.log.gz 2024-04-01T05:56:05.741 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.472045.log.gz 2024-04-01T05:56:05.766 INFO:teuthology.orchestra.run.smithi064.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.127701.log.gz 2024-04-01T05:56:05.768 INFO:teuthology.orchestra.run.smithi064.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.270814.log.gz 2024-04-01T05:56:05.776 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.380542.log.gz 2024-04-01T05:56:05.781 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.451478.log.gz 2024-04-01T05:56:05.819 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-04-01T05:56:05.822 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.214986.log.gz 2024-04-01T05:56:05.831 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.200203.log.gz 2024-04-01T05:56:05.834 INFO:teuthology.orchestra.run.smithi064.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.389837.log.gz 2024-04-01T05:56:05.836 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.352796.log.gz 2024-04-01T05:56:05.837 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.353658.log.gz 2024-04-01T05:56:05.842 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.118983.log.gz 2024-04-01T05:56:05.844 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.147537.log.gz 2024-04-01T05:56:05.862 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.392794.log.gz 2024-04-01T05:56:05.862 INFO:teuthology.orchestra.run.smithi064.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.139214.log.gz 2024-04-01T05:56:05.863 INFO:teuthology.orchestra.run.smithi064.stderr: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.191308.log.gz 2024-04-01T05:56:05.863 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.131100.log.gz 2024-04-01T05:56:05.890 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.432208.log.gz 2024-04-01T05:56:05.917 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.509176.log.gz 2024-04-01T05:56:05.950 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.507880.log.gz 2024-04-01T05:56:05.965 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.462861.log.gz 2024-04-01T05:56:05.968 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.465442.log.gz 2024-04-01T05:56:05.994 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.240035.log.gz 2024-04-01T05:56:05.994 INFO:teuthology.orchestra.run.smithi064.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.278521.log.gz 2024-04-01T05:56:05.999 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.524586.log.gz 2024-04-01T05:56:06.002 INFO:teuthology.orchestra.run.smithi064.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.533863.log.gz 2024-04-01T05:56:06.003 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.533030.log.gz 2024-04-01T05:56:06.006 INFO:teuthology.orchestra.run.smithi064.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.206478.log.gz 2024-04-01T05:56:06.008 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.483484.log.gz 2024-04-01T05:56:06.010 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.229580.log.gz 2024-04-01T05:56:06.011 INFO:teuthology.orchestra.run.smithi064.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.269512.log.gz 2024-04-01T05:56:06.013 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.272976.log.gz 2024-04-01T05:56:06.013 INFO:teuthology.orchestra.run.smithi064.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.138367.log.gz 2024-04-01T05:56:06.013 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.269521.log.gz 2024-04-01T05:56:06.014 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.137953.log.gz 2024-04-01T05:56:06.035 INFO:teuthology.orchestra.run.smithi064.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.143167.log.gz 2024-04-01T05:56:06.036 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.256672.log.gz 2024-04-01T05:56:06.036 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.225894.log.gz 2024-04-01T05:56:06.043 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.436026.log.gz 2024-04-01T05:56:06.046 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.499141.log.gz 2024-04-01T05:56:06.051 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.214086.log.gz 2024-04-01T05:56:06.060 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.132875.log.gz 2024-04-01T05:56:06.080 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.209850.log.gz 2024-04-01T05:56:06.080 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.269091.log.gz 2024-04-01T05:56:06.081 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.355058.log.gz 2024-04-01T05:56:06.083 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.266134.log.gz 2024-04-01T05:56:06.084 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.265722.log.gz 2024-04-01T05:56:06.105 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.423539.log.gz 2024-04-01T05:56:06.126 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47789.log.gz 2024-04-01T05:56:06.155 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-04-01T05:56:06.155 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.494316.log.gz 2024-04-01T05:56:06.157 INFO:teuthology.orchestra.run.smithi064.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.240475.log.gz 2024-04-01T05:56:06.165 INFO:teuthology.orchestra.run.smithi064.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.464567.log.gz 2024-04-01T05:56:06.167 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.472054.log.gz 2024-04-01T05:56:06.197 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.126397.log.gz 2024-04-01T05:56:06.199 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-04-01T05:56:06.201 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49382.log.gz 2024-04-01T05:56:06.201 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49424.log.gz 2024-04-01T05:56:06.209 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.349414.log.gz 2024-04-01T05:56:06.211 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.212383.log.gz 2024-04-01T05:56:06.212 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.415782.log.gz 2024-04-01T05:56:06.212 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.483035.log.gz 2024-04-01T05:56:06.257 INFO:teuthology.orchestra.run.smithi064.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.525841.log.gz 2024-04-01T05:56:06.257 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.527108.log.gz 2024-04-01T05:56:06.269 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.394229.log.gz 2024-04-01T05:56:06.281 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.504695.log.gz 2024-04-01T05:56:06.288 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.390767.log.gz 2024-04-01T05:56:06.293 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51690.log.gz 2024-04-01T05:56:06.295 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.160741.log.gz 2024-04-01T05:56:06.308 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.370417.log.gz 2024-04-01T05:56:06.310 INFO:teuthology.orchestra.run.smithi064.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.409607.log.gz 2024-04-01T05:56:06.311 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.424167.log.gz 2024-04-01T05:56:06.312 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.218053.log.gz 2024-04-01T05:56:06.324 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.509588.log.gz 2024-04-01T05:56:06.327 INFO:teuthology.orchestra.run.smithi064.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.156523.log.gz 2024-04-01T05:56:06.332 INFO:teuthology.orchestra.run.smithi064.stderr: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.263086.log.gz 2024-04-01T05:56:06.353 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.465009.log.gz 2024-04-01T05:56:06.381 INFO:teuthology.orchestra.run.smithi064.stderr: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.451756.log.gz 2024-04-01T05:56:06.384 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.344325.log.gz 2024-04-01T05:56:06.386 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.347284.log.gz 2024-04-01T05:56:06.394 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.190282.log.gz 2024-04-01T05:56:06.395 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.251517.log.gz 2024-04-01T05:56:06.435 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.134572.log.gz 2024-04-01T05:56:06.436 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.234468.log.gz 2024-04-01T05:56:06.438 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.521719.log.gz 2024-04-01T05:56:06.444 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.391279.log.gz 2024-04-01T05:56:06.451 INFO:teuthology.orchestra.run.smithi064.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.469943.log.gz 2024-04-01T05:56:06.455 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.154784.log.gz 2024-04-01T05:56:06.459 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-04-01T05:56:06.460 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.418649.log.gz 2024-04-01T05:56:06.464 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.392297.log.gz 2024-04-01T05:56:06.468 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.361486.log.gz 2024-04-01T05:56:06.469 INFO:teuthology.orchestra.run.smithi064.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.188904.log.gz 2024-04-01T05:56:06.474 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.367888.log.gz 2024-04-01T05:56:06.476 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.427903.log.gz 2024-04-01T05:56:06.485 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.408715.log.gz 2024-04-01T05:56:06.489 INFO:teuthology.orchestra.run.smithi064.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.502101.log.gz 2024-04-01T05:56:06.491 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.508320.log.gz 2024-04-01T05:56:06.492 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.219736.log.gz 2024-04-01T05:56:06.506 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.391784.log.gz 2024-04-01T05:56:06.509 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.201995.log.gz 2024-04-01T05:56:06.511 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.216339.log.gz 2024-04-01T05:56:06.512 INFO:teuthology.orchestra.run.smithi064.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.255403.log.gz 2024-04-01T05:56:06.512 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.231493.log.gz 2024-04-01T05:56:06.515 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.476529.log.gz 2024-04-01T05:56:06.526 INFO:teuthology.orchestra.run.smithi064.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.163799.log.gz 2024-04-01T05:56:06.548 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.347293.log.gz 2024-04-01T05:56:06.576 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.208175.log.gz 2024-04-01T05:56:06.578 INFO:teuthology.orchestra.run.smithi064.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.240906.log.gz 2024-04-01T05:56:06.586 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.200653.log.gz 2024-04-01T05:56:06.588 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.229589.log.gz 2024-04-01T05:56:06.592 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.531756.log.gz 2024-04-01T05:56:06.595 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.499562.log.gz 2024-04-01T05:56:06.595 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.532186.log.gz 2024-04-01T05:56:06.595 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.393762.log.gz 2024-04-01T05:56:06.596 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.363167.log.gz 2024-04-01T05:56:06.596 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.407844.log.gz 2024-04-01T05:56:06.596 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.409140.log.gz 2024-04-01T05:56:06.597 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48700.log.gz 2024-04-01T05:56:06.597 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.491318.log.gz 2024-04-01T05:56:06.598 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.410037.log.gz 2024-04-01T05:56:06.598 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-04-01T05:56:06.626 INFO:teuthology.orchestra.run.smithi064.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.528373.log.gz 2024-04-01T05:56:06.627 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.528382.log.gz 2024-04-01T05:56:06.631 INFO:teuthology.orchestra.run.smithi064.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.251058.log.gz 2024-04-01T05:56:06.642 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.256259.log.gz 2024-04-01T05:56:06.644 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.436035.log.gz 2024-04-01T05:56:06.669 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.482436.log.gz 2024-04-01T05:56:06.707 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.260518.log.gz 2024-04-01T05:56:06.713 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-01T05:56:06.720 INFO:teuthology.orchestra.run.smithi064.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.488107.log.gz 2024-04-01T05:56:06.723 INFO:teuthology.orchestra.run.smithi064.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.121143.log.gz 2024-04-01T05:56:06.728 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.376085.log.gz 2024-04-01T05:56:06.737 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.490464.log.gz 2024-04-01T05:56:06.758 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.183507.log.gz 2024-04-01T05:56:06.766 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.192136.log.gz 2024-04-01T05:56:06.769 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.487538.log.gz 2024-04-01T05:56:06.778 INFO:teuthology.orchestra.run.smithi064.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.510039.log.gz 2024-04-01T05:56:06.780 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.179234.log.gz 2024-04-01T05:56:06.787 INFO:teuthology.orchestra.run.smithi064.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.392785.log.gz 2024-04-01T05:56:06.840 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.388056.log.gz 2024-04-01T05:56:06.840 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.228235.log.gz 2024-04-01T05:56:06.843 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.390282.log.gz 2024-04-01T05:56:06.845 INFO:teuthology.orchestra.run.smithi064.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.519000.log.gz 2024-04-01T05:56:06.861 INFO:teuthology.orchestra.run.smithi064.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.421801.log.gz 2024-04-01T05:56:06.884 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.434329.log.gz 2024-04-01T05:56:06.887 INFO:teuthology.orchestra.run.smithi064.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.384853.log.gz 2024-04-01T05:56:06.892 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.478286.log.gz 2024-04-01T05:56:07.180 INFO:teuthology.orchestra.run.smithi064.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.205147.log.gz 2024-04-01T05:56:07.180 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.499132.log.gz 2024-04-01T05:56:07.181 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.495258.log.gz 2024-04-01T05:56:07.182 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.211969.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% 90.0% -- replaced with /var/log/ceph/ceph-client.admin.120309.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.261788.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45989.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.474314.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.406933.log.gz -- replaced with /var/log/ceph/ceph-client.admin.389828.log.gz 2024-04-01T05:56:07.380 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.394238.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.362755.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.356402.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 79.9% 77.7% -- replaced with /var/log/ceph/ceph-client.admin.484761.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.216760.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.118992.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.515046.log.gz 2024-04-01T05:56:07.381 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.421379.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.135405.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51020.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.211114.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.506123.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.450601.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.239581.log.gz 2024-04-01T05:56:07.382 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.423798.log.gz 2024-04-01T05:56:07.420 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.258809.log.gz 2024-04-01T05:56:07.495 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.346030.log.gz 2024-04-01T05:56:07.502 INFO:teuthology.orchestra.run.smithi064.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.468210.log.gz 2024-04-01T05:56:07.544 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.378387.log.gz 2024-04-01T05:56:07.784 INFO:teuthology.orchestra.run.smithi064.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.345173.log.gz 2024-04-01T05:56:07.835 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.448878.log.gz 2024-04-01T05:56:07.925 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.494773.log.gz 2024-04-01T05:56:07.926 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.236153.log.gz 2024-04-01T05:56:07.927 INFO:teuthology.orchestra.run.smithi064.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.279895.log.gz 2024-04-01T05:56:07.958 INFO:teuthology.orchestra.run.smithi064.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.414933.log.gz 2024-04-01T05:56:08.054 INFO:teuthology.orchestra.run.smithi064.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.206903.log.gz 2024-04-01T05:56:08.065 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.247812.log.gz 2024-04-01T05:56:08.084 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-04-01T05:56:08.090 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.156532.log.gz 2024-04-01T05:56:08.116 INFO:teuthology.orchestra.run.smithi064.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.190800.log.gz 2024-04-01T05:56:08.125 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.190273.log.gz 2024-04-01T05:56:08.182 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.259244.log.gz 2024-04-01T05:56:08.200 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.266555.log.gz 2024-04-01T05:56:08.227 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.203805.log.gz 2024-04-01T05:56:08.374 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45534.log.gz 2024-04-01T05:56:08.500 INFO:teuthology.orchestra.run.smithi064.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.203796.log.gz 2024-04-01T05:56:08.534 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.359814.log.gz 2024-04-01T05:56:08.589 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-01T05:56:08.658 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.387584.log.gz 2024-04-01T05:56:08.667 INFO:teuthology.orchestra.run.smithi064.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.381000.log.gz 2024-04-01T05:56:08.688 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.278959.log.gz 2024-04-01T05:56:08.726 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.417520.log.gz 2024-04-01T05:56:08.750 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.364439.log.gz 2024-04-01T05:56:08.781 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.235311.log.gz 2024-04-01T05:56:08.786 INFO:teuthology.orchestra.run.smithi064.stderr: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.383160.log.gz 2024-04-01T05:56:08.788 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.383169.log.gz 2024-04-01T05:56:08.810 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.267406.log.gz 2024-04-01T05:56:08.811 INFO:teuthology.orchestra.run.smithi064.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.420486.log.gz 2024-04-01T05:56:08.813 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.475651.log.gz 2024-04-01T05:56:08.842 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.417062.log.gz 2024-04-01T05:56:08.842 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-01T05:56:08.914 INFO:teuthology.orchestra.run.smithi064.stderr: 79.2% -- replaced with /var/log/ceph/ceph-client.admin.226306.log.gz 2024-04-01T05:56:08.917 INFO:teuthology.orchestra.run.smithi064.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.390758.log.gz 2024-04-01T05:56:08.940 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.188450.log.gz 2024-04-01T05:56:09.054 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.443292.log.gz 2024-04-01T05:56:09.272 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.164075.log.gz 2024-04-01T05:56:09.412 INFO:teuthology.orchestra.run.smithi064.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.410910.log.gz 2024-04-01T05:56:09.413 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.230067.log.gz 2024-04-01T05:56:09.414 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47915.log.gz 2024-04-01T05:56:09.417 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.280852.log.gz 2024-04-01T05:56:09.419 INFO:teuthology.orchestra.run.smithi064.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.407409.log.gz 2024-04-01T05:56:09.431 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.190809.log.gz 2024-04-01T05:56:09.435 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.423846.log.gz 2024-04-01T05:56:09.536 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.163502.log.gz 2024-04-01T05:56:09.547 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51476.log.gz 2024-04-01T05:56:09.550 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.138376.log.gz 2024-04-01T05:56:09.565 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.262223.log.gz 2024-04-01T05:56:09.780 INFO:teuthology.orchestra.run.smithi064.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.393771.log.gz 2024-04-01T05:56:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.472481.log.gz 2024-04-01T05:56:10.086 INFO:teuthology.orchestra.run.smithi064.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.258388.log.gz 2024-04-01T05:56:10.087 INFO:teuthology.orchestra.run.smithi064.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.260936.log.gz 2024-04-01T05:56:10.365 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.430497.log.gz 2024-04-01T05:56:10.553 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2024-04-01T05:56:10.599 INFO:teuthology.orchestra.run.smithi064.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.192969.log.gz 2024-04-01T05:56:10.653 INFO:teuthology.orchestra.run.smithi064.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.424176.log.gz 2024-04-01T05:56:10.782 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.355899.log.gz 2024-04-01T05:56:11.047 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.225465.log.gz 2024-04-01T05:56:13.172 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% 93.3% -- replaced with /var/log/ceph/ceph-client.admin.196632.log.gz 2024-04-01T05:56:13.172 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.410459.log.gz 2024-04-01T05:56:13.172 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.374662.log.gz 2024-04-01T05:56:13.173 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.409149.log.gz 2024-04-01T05:56:13.173 INFO:teuthology.orchestra.run.smithi064.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.222876.log.gz 2024-04-01T05:56:13.173 INFO:teuthology.orchestra.run.smithi064.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.111673.log.gz 2024-04-01T05:56:13.892 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.423894.log.gz 2024-04-01T05:56:15.302 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.113162.log.gz 2024-04-01T05:56:19.262 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.115504.log.gz 2024-04-01T05:56:19.710 INFO:teuthology.orchestra.run.smithi064.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.562266.log.gz 2024-04-01T05:56:20.486 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-01T05:56:21.582 INFO:teuthology.orchestra.run.smithi064.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.411334.log.gz 2024-04-01T05:56:21.652 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.125645.log.gz 2024-04-01T05:56:21.685 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-01T05:56:21.844 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-01T05:56:23.055 INFO:teuthology.orchestra.run.smithi100.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-01T05:56:27.918 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.419070.log.gz 2024-04-01T05:56:28.143 INFO:teuthology.orchestra.run.smithi100.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-01T05:56:33.770 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.124695.log.gz 2024-04-01T05:56:35.507 INFO:teuthology.orchestra.run.smithi100.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-01T05:56:35.618 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.353667.log.gz 2024-04-01T05:56:35.969 INFO:teuthology.orchestra.run.smithi064.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.262644.log.gz 2024-04-01T05:56:37.889 INFO:teuthology.orchestra.run.smithi064.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.227776.log.gz 2024-04-01T05:56:39.460 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.386200.log.gz 2024-04-01T05:56:40.256 INFO:teuthology.orchestra.run.smithi064.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.250433.log.gz 2024-04-01T05:56:41.847 INFO:teuthology.orchestra.run.smithi064.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.128131.log.gz 2024-04-01T05:56:41.941 INFO:teuthology.orchestra.run.smithi064.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.354185.log.gz 2024-04-01T05:56:43.112 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.141817.log.gz 2024-04-01T05:56:44.567 INFO:teuthology.orchestra.run.smithi064.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.116456.log.gz 2024-04-01T05:56:44.923 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.230608.log.gz 2024-04-01T05:56:45.890 INFO:teuthology.orchestra.run.smithi064.stderr: 95.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-01T05:56:46.317 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.142285.log.gz 2024-04-01T05:56:47.036 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.379231.log.gz 2024-04-01T05:56:47.362 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.249988.log.gz 2024-04-01T05:56:47.449 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.140923.log.gz 2024-04-01T05:56:48.225 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.428338.log.gz 2024-04-01T05:56:48.227 INFO:teuthology.orchestra.run.smithi064.stderr: 97.3% -- replaced with /var/log/ceph/ceph-client.admin.115073.log.gz 2024-04-01T05:56:49.358 INFO:teuthology.orchestra.run.smithi100.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-01T05:56:49.752 INFO:teuthology.orchestra.run.smithi064.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.admin.280395.log.gz 2024-04-01T05:56:50.030 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.353231.log.gz 2024-04-01T05:56:50.371 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.125170.log.gz 2024-04-01T05:56:51.674 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.249512.log.gz 2024-04-01T05:56:51.718 INFO:teuthology.orchestra.run.smithi064.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.114642.log.gz 2024-04-01T05:56:52.100 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.143176.log.gz 2024-04-01T05:56:52.226 INFO:teuthology.orchestra.run.smithi064.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.482445.log.gz 2024-04-01T05:56:52.494 INFO:teuthology.orchestra.run.smithi064.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.147084.log.gz 2024-04-01T05:56:53.362 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.461558.log.gz 2024-04-01T05:56:58.813 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.119413.log.gz 2024-04-01T05:56:58.821 INFO:teuthology.orchestra.run.smithi064.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.226736.log.gz 2024-04-01T05:57:00.404 INFO:teuthology.orchestra.run.smithi064.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.417942.log.gz 2024-04-01T05:57:05.325 INFO:teuthology.orchestra.run.smithi100.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-01T05:57:11.856 INFO:teuthology.orchestra.run.smithi064.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.121152.log.gz 2024-04-01T05:57:16.220 INFO:teuthology.orchestra.run.smithi064.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.246939.log.gz 2024-04-01T05:57:20.202 INFO:teuthology.orchestra.run.smithi064.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.505116.log.gz 2024-04-01T05:57:20.890 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.420020.log.gz 2024-04-01T05:57:37.912 INFO:teuthology.orchestra.run.smithi064.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.244499.log.gz 2024-04-01T05:57:56.827 INFO:teuthology.orchestra.run.smithi064.stderr: 97.8% -- replaced with /var/log/ceph/ceph-client.admin.502952.log.gz 2024-04-01T05:58:02.619 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-01T05:58:02.619 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-04-01T05:58:02.619 INFO:teuthology.orchestra.run.smithi100.stderr:real 2m0.679s 2024-04-01T05:58:02.619 INFO:teuthology.orchestra.run.smithi100.stderr:user 3m10.594s 2024-04-01T05:58:02.620 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m8.161s 2024-04-01T05:58:30.377 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.118323.log.gz 2024-04-01T05:58:35.148 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.183966.log.gz 2024-04-01T05:58:46.440 INFO:teuthology.orchestra.run.smithi064.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.253203.log.gz 2024-04-01T05:58:47.234 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.151294.log.gz 2024-04-01T05:58:48.156 INFO:teuthology.orchestra.run.smithi064.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.371285.log.gz 2024-04-01T05:59:39.351 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.admin.131570.log.gz 2024-04-01T05:59:51.345 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.184466.log.gz 2024-04-01T06:00:21.964 INFO:teuthology.orchestra.run.smithi064.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.388917.log.gz 2024-04-01T06:00:26.816 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.160750.log.gz 2024-04-01T06:00:43.718 INFO:teuthology.orchestra.run.smithi064.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.227194.log.gz 2024-04-01T06:02:40.031 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.164946.log.gz 2024-04-01T06:03:11.206 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.146197.log.gz 2024-04-01T06:03:54.668 INFO:teuthology.orchestra.run.smithi064.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.194282.log.gz 2024-04-01T06:03:56.753 INFO:teuthology.orchestra.run.smithi064.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.113672.log.gz 2024-04-01T06:04:40.365 INFO:teuthology.orchestra.run.smithi064.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.420920.log.gz 2024-04-01T06:06:36.096 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.115969.log.gz 2024-04-01T06:07:42.207 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-01T06:09:47.764 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.524295.log.gz 2024-04-01T06:11:19.452 INFO:teuthology.orchestra.run.smithi064.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.395684.log.gz 2024-04-01T06:12:13.801 INFO:teuthology.orchestra.run.smithi064.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.263095.log.gz 2024-04-01T06:12:29.626 INFO:teuthology.orchestra.run.smithi064.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.114154.log.gz 2024-04-01T06:14:08.194 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.116947.log.gz 2024-04-01T06:17:14.055 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.515055.log.gz 2024-04-01T06:17:28.799 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.516412.log.gz 2024-04-01T06:19:01.992 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.241775.log.gz 2024-04-01T06:19:21.862 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.513760.log.gz 2024-04-01T06:19:22.959 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.522979.log.gz 2024-04-01T06:19:36.704 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.521728.log.gz 2024-04-01T06:19:40.396 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.519009.log.gz 2024-04-01T06:19:40.926 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.510927.log.gz 2024-04-01T06:19:48.958 INFO:teuthology.orchestra.run.smithi064.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.517791.log.gz 2024-04-01T06:19:58.121 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.520302.log.gz 2024-04-01T06:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.512513.log.gz 2024-04-01T06:25:03.692 INFO:teuthology.orchestra.run.smithi064.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.191317.log.gz 2024-04-01T06:26:14.872 INFO:teuthology.orchestra.run.smithi064.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.283395.log.gz 2024-04-01T06:28:34.462 INFO:teuthology.orchestra.run.smithi064.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.453815.log.gz 2024-04-01T06:31:51.771 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-01T06:37:06.007 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.445625.log.gz 2024-04-01T06:38:19.987 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.443718.log.gz 2024-04-01T06:39:32.342 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-01T06:42:38.938 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.273440.log.gz 2024-04-01T06:44:22.964 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.157454.log.gz 2024-04-01T06:54:20.938 INFO:teuthology.orchestra.run.smithi064.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.147963.log.gz 2024-04-01T07:14:41.165 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.535195.log.gz 2024-04-01T07:14:41.194 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-01T07:14:41.195 INFO:teuthology.orchestra.run.smithi064.stderr:real 78m39.258s 2024-04-01T07:14:41.195 INFO:teuthology.orchestra.run.smithi064.stderr:user 58m19.924s 2024-04-01T07:14:41.195 INFO:teuthology.orchestra.run.smithi064.stderr:sys 2m22.699s 2024-04-01T07:14:41.195 INFO:tasks.ceph:Archiving logs... 2024-04-01T07:14:41.197 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi064/log 2024-04-01T07:14:41.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-01T07:21:01.304 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi100/log 2024-04-01T07:21:01.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-01T07:27:29.473 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-01T07:27:29.484 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-01T07:27:29.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-01T07:27:29.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-01T07:27:29.595 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-01T07:27:29.604 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-01T07:27:29.604 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-01T07:27:29.607 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-01T07:27:29.621 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-04-01T07:27:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-04-01T07:27:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-01T07:27:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-04-01T07:27:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +57us[ +81us] +/- 67ms 2024-04-01T07:27:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 7 377 16 -337us[ -312us] +/- 56ms 2024-04-01T07:27:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 120 +575us[ +598us] +/- 90ms 2024-04-01T07:27:29.688 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 75 +41us[ +25us] +/- 67ms 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -457us[ -473us] +/- 56ms 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 51 +524us[ +524us] +/- 91ms 2024-04-01T07:27:29.717 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-01T07:27:29.719 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-01T07:27:29.728 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-01T07:27:29.729 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-01T07:27:29.752 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-01T07:27:30.006 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-01T07:27:30.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-01T07:27:30.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T07:27:30.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-01T07:27:30.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T07:27:30.205 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-01T07:27:30.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-01T07:27:30.252 INFO:teuthology.task.internal:Duration was 26450.240940 seconds 2024-04-01T07:27:30.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-01T07:27:30.286 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-01T07:27:30.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-01T07:27:30.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-01T07:27:30.373 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-01T07:27:30.571 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-01T07:27:31.084 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-01T07:27:31.085 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-04-01T07:27:31.085 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-01T07:27:31.131 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-04-01T07:27:31.131 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-01T07:27:31.152 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-01T07:27:31.152 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-01T07:27:31.155 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-01T07:27:31.221 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-01T07:27:31.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-01T07:27:31.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-01T07:27:32.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-01T07:27:32.437 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-01T07:27:32.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-01T07:27:32.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-01T07:27:32.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-01T07:27:32.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-01T07:27:32.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-01T07:27:32.615 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-04-01T07:27:32.622 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-04-01T07:27:32.697 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-01T07:27:32.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T07:27:32.729 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-01T07:27:32.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-01T07:27:32.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-01T07:27:32.761 INFO:teuthology.task.internal:Transferring archived files... 2024-04-01T07:27:32.762 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi064 2024-04-01T07:27:32.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-01T07:27:32.852 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198/remote/smithi100 2024-04-01T07:27:32.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-01T07:27:32.892 INFO:teuthology.task.internal:Removing archive directory... 2024-04-01T07:27:32.892 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-01T07:27:32.896 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-01T07:27:32.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-01T07:27:32.953 INFO:teuthology.task.internal:Not uploading archives. 2024-04-01T07:27:32.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-01T07:27:32.978 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-01T07:27:32.978 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-01T07:27:32.981 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-01T07:27:32.996 INFO:teuthology.orchestra.run.smithi100.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 1 07:27 /home/ubuntu/cephtest 2024-04-01T07:27:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: 266042 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 1 07:27 /home/ubuntu/cephtest 2024-04-01T07:27:33.016 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-01T07:27:33.027 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-01T07:27:33.086 INFO:teuthology.nuke:Checking targets against current locks 2024-04-01T07:27:33.113 DEBUG:teuthology.nuke:shortname: smithi064 2024-04-01T07:27:33.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-01T07:27:33.138 DEBUG:teuthology.nuke:shortname: smithi100 2024-04-01T07:27:33.139 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-01T07:27:33.143 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-31 23:58:26.933271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO47IA+EvSR0YgNt5WX/u0uLqP/LctJOxCRWkyqMGKRxSM5C6a6eiCyA2wcSO6kp0CNMI7Qqft9C11I/X8lVYQ='} 2024-04-01T07:27:33.164 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-31 23:58:26.934688', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoLc4FeywrK/SqV54UHq0a0Sj8ZFU63fFNKCIdmYT78xPktjt7ncDU1VQRYkEOPbhv+QsYrsbm8EXPM5SvTa0I='} 2024-04-01T07:27:33.196 INFO:teuthology.orchestra.console.smithi064:Power off 2024-04-01T07:27:33.196 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T07:27:33.222 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-04-01T07:27:33.222 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:33.239 INFO:teuthology.orchestra.console.smithi100:Power off 2024-04-01T07:27:33.240 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T07:27:33.249 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-04-01T07:27:33.262 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-04-01T07:27:33.262 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:33.287 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-01T07:27:37.251 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:37.289 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:37.364 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-04-01T07:27:37.402 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-01T07:27:41.367 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:41.403 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T07:27:41.478 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-04-01T07:27:41.515 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-01T07:27:41.578 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-04-01T07:27:41.616 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-04-01T07:27:41.721 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} duration: 26450.240940332413 failure_reason: 'Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev)' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8ac1191bb37341bb9199a6c70ad548cb status: fail success: false 2024-04-01T07:27:41.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T07:27:41.804 INFO:teuthology.run:FAIL